Review Board 1.7.22


HBASE-2233 Support both Hadoop 0.20, 0.21, and 0.22

Review Request #858 - Created June 6, 2011 and updated

Michael Stack
HBASE-2233
Reviewers
hbase
hbase-git
Patch for HBASE TRUNK that makes it so we run on hadoop 0.20.x or hadoop 0.22 branch (as it is now).

Needed hdfs-1948 and hadoop-7531 applied to hadoop 0.22 branch.  Because these issues are NOT in hadoop 0.21, hbase will not work against 0.21 hadoop, not w/o these patches and a rebuild.

Ran this patch with hadoop 0.20 and hadoop 0.22 underneath hbase (I used the patch for the pom that is attached to the issue to switch to hadoop 0.22).  All unit tests seem to pass except TestHFileOutputFormat when up on hadoop 0.22; hadoop 0.22 uses avro 1.3.2 whereas hbase is up on avro 1.4.1.  The test fails with a method not found exception.  You need to enable mapreduce logging to see the exception.
Posted (June 6, 2011, 10:51 p.m.)
nice work :) just some small comments.
is this change related?
  1. Not related but needed.  I could do in separate issue.   Saw ugly NPE when stuff came up ugly when not patched properly.
also this
hrm, I dont understand this line
  1. Good one.  Test should be inverted.  Fixing...
did our behavior here change? I think we used to return null, but I thought now we returned an empty list?
  1. fs.listStatus changed in that if you pass it a dir that does not exist, before it would return empty list but no throws FileNotFoundException.
    
    Regards our behavior changing around the return out of this method, I don't think so.  I looked at 0.90 and it does this too:
    
        if (files == null) return filesSorted;
    
    ... returning empty NavigableSet if nothing found.
same q here
CONFI?
  1. Yeah.  Takes variable named 'conf1'.  I changed it to be less surprising CONF_WITH_LOCALFS
Ship it!
Posted (June 7, 2011, 5:09 p.m.)
sounds good, +1 after fixing that condition