Review Board 1.7.22


Fix for SQOOP-934: Potential connection timeout

Review Request #11940 - Created June 18, 2013 and submitted

Raghav Gautam
SQOOP-934
Reviewers
Sqoop
sqoop-trunk
This patches fixes the SQOOP-934. The connection created prior to map-reduce job can potentially timed out while the job is running. This will cause problems when this connection is reused for generating the hive script. This patch discards the connection prior to hive script generation. So, a fresh connection will be used for generating hive script.
Unit tests pass.
Manually tested hive import.
src/java/org/apache/sqoop/hive/HiveImport.java
Revision 02596a6 New Change
[20] 176 lines
[+20] [+] public void importTable(String inputTableName, String outputTableName,
177
      env.add("EXPECTED_SCRIPT=" + expectedScript);
177
      env.add("EXPECTED_SCRIPT=" + expectedScript);
178
      env.add("TMPDIR=" + options.getTempDir());
178
      env.add("TMPDIR=" + options.getTempDir());
179
    }
179
    }
180

    
   
180

   
181
    // generate the HQL statements to run.
181
    // generate the HQL statements to run.

    
   
182
    // reset the connection as it might have timed out

    
   
183
    connManager.discardConnection(true);
182
    TableDefWriter tableWriter = new TableDefWriter(options, connManager,
184
    TableDefWriter tableWriter = new TableDefWriter(options, connManager,
183
        inputTableName, outputTableName,
185
        inputTableName, outputTableName,
184
        configuration, !debugMode);
186
        configuration, !debugMode);
185
    String createTableStr = tableWriter.getCreateTableStmt() + ";\n";
187
    String createTableStr = tableWriter.getCreateTableStmt() + ";\n";
186
    String loadDataStmtStr = tableWriter.getLoadDataStmt() + ";\n";
188
    String loadDataStmtStr = tableWriter.getLoadDataStmt() + ";\n";
[+20] [20] 201 lines
src/java/org/apache/sqoop/manager/ConnManager.java
Revision 32e736c New Change
 
src/java/org/apache/sqoop/manager/GenericJdbcManager.java
Revision 233e546 New Change
 
  1. src/java/org/apache/sqoop/hive/HiveImport.java: Loading...
  2. src/java/org/apache/sqoop/manager/ConnManager.java: Loading...
  3. src/java/org/apache/sqoop/manager/GenericJdbcManager.java: Loading...