Review Board 1.7.22


SQOOP-1032: Add the --bulk-load-dir option to support the HBase doBulkLoad function

Review Request #14240 - Created Sept. 19, 2013 and updated

Alexandre Normand
Reviewers
Sqoop
jarcec, jmspaggi, vasanthkumar
sqoop-trunk
This is the patch to address some of the comments on Zhancheng Deng's review request (https://reviews.apache.org/r/13052/) which was basically the change to add the --bulk-load-dir option to support the HBase doBulkLoad function.

 
src/docs/user/hbase.txt
Revision 34f9875 New Change
[20] 55 lines
[+20]
56
to its string representation (as if you were importing to HDFS in text
56
to its string representation (as if you were importing to HDFS in text
57
mode), and then inserts the UTF-8 bytes of this string in the target
57
mode), and then inserts the UTF-8 bytes of this string in the target
58
cell. Sqoop will skip all rows containing null values in all columns
58
cell. Sqoop will skip all rows containing null values in all columns
59
except the row key column.
59
except the row key column.
60

    
   
60

   
61

    
   
61
To decrease the load on hbase, Sqoop can do bulk loading as opposed to

    
   
62
direct writes. To use bulk loading, enable it using +\--hbase-bulkload+.
src/docs/user/hbase-args.txt
Revision 8ba23eb New Change
 
src/java/org/apache/sqoop/SqoopOptions.java
Revision 01805f9 New Change
 
src/java/org/apache/sqoop/hbase/HBasePutProcessor.java
Revision 9ceb5bd New Change
 
src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java
Revision 5ccf311 New Change
 
src/java/org/apache/sqoop/manager/SqlManager.java
Revision 2a4992d New Change
 
src/java/org/apache/sqoop/mapreduce/HBaseBulkImportJob.java
New File
 
src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java
New File
 
src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
Revision ab7f21e New Change
 
src/java/org/apache/sqoop/tool/BaseSqoopTool.java
Revision 0eca991 New Change
 
src/test/com/cloudera/sqoop/TestSqoopOptions.java
Revision 03e2504 New Change
 
  1. src/docs/user/hbase.txt: Loading...
  2. src/docs/user/hbase-args.txt: Loading...
  3. src/java/org/apache/sqoop/SqoopOptions.java: Loading...
  4. src/java/org/apache/sqoop/hbase/HBasePutProcessor.java: Loading...
  5. src/java/org/apache/sqoop/hbase/ToStringPutTransformer.java: Loading...
  6. src/java/org/apache/sqoop/manager/SqlManager.java: Loading...
  7. src/java/org/apache/sqoop/mapreduce/HBaseBulkImportJob.java: Loading...
  8. src/java/org/apache/sqoop/mapreduce/HBaseBulkImportMapper.java: Loading...
  9. src/java/org/apache/sqoop/mapreduce/ImportJobBase.java: Loading...
  10. src/java/org/apache/sqoop/tool/BaseSqoopTool.java: Loading...
  11. src/test/com/cloudera/sqoop/TestSqoopOptions.java: Loading...