float and double calculation is inaccurate in Hive
Review Request #8653 - Created Dec. 18, 2012 and updated
I found this during debug the e2e test failures. I found Hive miss calculate the float and double value. Take float calculation as an example: hive> select f from all100k limit 1; 48308.98 hive> select f/10 from all100k limit 1; 4830.898046875 <--added 04875 in the end hive> select f*1.01 from all100k limit 1; 48792.0702734375 <--should be 48792.0698 It might be essentially the same problem as http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm But since e2e test compare the results with mysql and seems mysql does it right, so it is worthy fixing it in Hive.
I did test to compare the result with mysql default float precision setting, the result is identical. query: select f, f*1.01, f/10 from all100k limit 1; mysql result: 48309 48792.0702734375 4830.898046875 hive result: 48308.98 48792.0702734375 4830.898046875 I apply this patch and run the hive e2e test, and the tests all pass (without this patch, 5 related failures)