Review Board 1.7.22


HIVE-4957: Restrict number of bit vectors, to prevent out of Java heap memory

Review Request #14250 - Created Sept. 20, 2013 and updated

Shreepadma Venugopalan
trunk
HIVE-4957
Reviewers
hive
brocknoland
hive-git
Restricts the number of bit vectors used by Flajolet-Martin distinct value estimator to 1024.
Adds a new negative test case.
Total:
3
Open:
3
Resolved:
0
Dropped:
0
Status:
From:
Posted (Sept. 20, 2013, 8:38 p.m.)

   

  
Please use a static variable instead of repeating "1024":

private static final MAX_NUM_BIT_VECTORS = 1024;
  1. 1024 is not repeated within a class. However, we can repeat the constant declaration across classes instead.
Why should it raise an error?
  1. Added a longer comment.
Posted (Sept. 21, 2013, 2:27 a.m.)

   

  
The error message isn't making it back to the user because it's getting generated at runtime on the cluster. Is it possible to bounds check this parameter at compile time instead?
  1. Today, the UDF framework doesn't support validation of inputs at compile time. Given the current framework, this is the best we can do. 
Ship it!
Posted (Oct. 18, 2013, 7:49 p.m.)
Ship It!