Review Board 1.7.22


SQOOP-690. Fix threading issues in SqoopOutputFormatLoadExecutor

Review Request #8063 - Created Nov. 15, 2012 and updated

Hari Shreedharan
SQOOP-690
Reviewers
Sqoop
sqoop-sqoop2
Improved thread handling in SqoopOutputFormatLoadExecutor. Removed the synchronized blocks and wait/notify.
Ran unit tests, and on a real cluster.
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
Revision ff7384c New Change
[20] 26 lines
[+20] [+] public class GenericJdbcExportLoader extends Loader {
27
  public static final int DEFAULT_BATCHES_PER_TRANSACTION = 100;
27
  public static final int DEFAULT_BATCHES_PER_TRANSACTION = 100;
28
  private int rowsPerBatch = DEFAULT_ROWS_PER_BATCH;
28
  private int rowsPerBatch = DEFAULT_ROWS_PER_BATCH;
29
  private int batchesPerTransaction = DEFAULT_BATCHES_PER_TRANSACTION;
29
  private int batchesPerTransaction = DEFAULT_BATCHES_PER_TRANSACTION;
30

    
   
30

   
31
  @Override
31
  @Override
32
  public void run(ImmutableContext context, DataReader reader) {
32
  public void run(ImmutableContext context, DataReader reader) throws Exception{
33
    String driver = context.getString(
33
    String driver = context.getString(
34
        GenericJdbcConnectorConstants.CONNECTOR_JDBC_DRIVER);
34
        GenericJdbcConnectorConstants.CONNECTOR_JDBC_DRIVER);
35
    String url = context.getString(
35
    String url = context.getString(
36
        GenericJdbcConnectorConstants.CONNECTOR_JDBC_URL);
36
        GenericJdbcConnectorConstants.CONNECTOR_JDBC_URL);
37
    String username = context.getString(
37
    String username = context.getString(
[+20] [20] 43 lines
execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java
Revision 29a73b0 New Change
 
execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java
Revision 711df0f New Change
 
execution/mapreduce/src/main/java/org/apache/sqoop/job/io/Data.java
Revision 41fceb8 New Change
 
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java
Revision 0d636ae New Change
 
execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java
Revision 6dcf784 New Change
 
spi/src/main/java/org/apache/sqoop/job/etl/Loader.java
Revision 3a708df New Change
 
spi/src/main/java/org/apache/sqoop/job/io/DataReader.java
Revision 18e2fb7 New Change
 
  1. connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java: Loading...
  2. execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsSequenceImportLoader.java: Loading...
  3. execution/mapreduce/src/main/java/org/apache/sqoop/job/etl/HdfsTextImportLoader.java: Loading...
  4. execution/mapreduce/src/main/java/org/apache/sqoop/job/io/Data.java: Loading...
  5. execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java: Loading...
  6. execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java: Loading...
  7. spi/src/main/java/org/apache/sqoop/job/etl/Loader.java: Loading...
  8. spi/src/main/java/org/apache/sqoop/job/io/DataReader.java: Loading...