Review Board 1.7.22


float and double calculation is inaccurate in Hive

Review Request #8653 - Created Dec. 18, 2012 and updated

Johnny Zhang
trunk
HIVE-3715
Reviewers
hive
hive
I found this during debug the e2e test failures. I found Hive miss calculate the float and double value. Take float calculation as an example:
hive> select f from all100k limit 1;
48308.98
hive> select f/10 from all100k limit 1;
4830.898046875 <--added 04875 in the end
hive> select f*1.01 from all100k limit 1;
48792.0702734375 <--should be 48792.0698
It might be essentially the same problem as http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm But since e2e test compare the results with mysql and seems mysql does it right, so it is worthy fixing it in Hive.
I did test to compare the result with mysql default float precision setting, the result is identical.

query:          select f, f*1.01, f/10 from all100k limit 1;
mysql result:   48309       48792.0702734375    4830.898046875
hive result:    48308.98    48792.0702734375	4830.898046875


I apply this patch and run the hive e2e test, and the tests all pass (without this patch, 5 related failures)
Total:
2
Open:
0
Resolved:
2
Dropped:
0
Status:
From:
Review request changed
Updated (Dec. 18, 2012, 12:37 a.m.)
  • I found this during debug the e2e test failures. I found Hive miss calculate the float and double value. Take float calculation as an example:
    hive> select f from all100k limit 1;
    48308.98
    hive> select f/10 from all100k limit 1;
    4830.898046875 <--added 04875 in the end
    hive> select f*1.01 from all100k limit 1;
    48792.0702734375 <--should be 48792.0698
    It might be essentially the same problem as http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm. But since e2e test compare the results with mysql and seems mysql does it right, so it is worthy fixing it in Hive.

    I found this during debug the e2e test failures. I found Hive miss calculate the float and double value. Take float calculation as an example:
    hive> select f from all100k limit 1;
    48308.98
    hive> select f/10 from all100k limit 1;
    4830.898046875 <--added 04875 in the end
    hive> select f*1.01 from all100k limit 1;
    48792.0702734375 <--should be 48792.0698
    It might be essentially the same problem as http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm But since e2e test compare the results with mysql and seems mysql does it right, so it is worthy fixing it in Hive.
Posted (Dec. 18, 2012, 12:38 a.m.)

   

  
10 seems to be a rather arbitrary number for scale. Any particular reason you are using it? Maybe we should invoke the method where no scale needs to be specified.
  1. Hi, Mark, thanks for reviewing it. The reason using 10 is because it is the same as mysql default precision setting. Just want to make the calculation result identical to mysql's
  2. I think I did tried without specify scale, and the result is different from mysql. I agree hard coding the scale is not a good way. Open to other suggestions.
  3. Fair enough. Thanks
You seem to be doing
DoubleWritable->String->BigDecimal

There probably is a way to do:
DoubleWritable->Double->BigDecimal

I am not sure if it's any more efficient the present case. So, take this suggestion with a grain of salt:-)
  1. the reason using constructor with String parameter is because using constructor with double parameter would reduce the precision before calculation. There is a similar discussion regarding it http://www.coderanch.com/t/408226/java/java/Double-BigDecimal-Conversion-problems
    
    "you will see the difference between creating an instance using a double (whose precision has already been compromised by forcing it into IEEE 754 standards) and creating an instance using a String (which can be translated accurately). "