Review Board 1.7.22


SQOOP-621: Requesting support for upsert export with MySQL

Review Request #7588 - Created Oct. 15, 2012 and submitted

Jarek Cecho
SQOOP-621
Reviewers
Sqoop
sqoop-trunk
I've implemented upsert functionality using MySQL clause INSERT INTO ... ON DUPLICATE KEY UPDATE. This clause have slightly different purpose than Oracle's MERGE statement and therefore the functionality is slightly different. I've provided warning message notifying user that column names specified in --update-key parameter are not going to be used.
I've added new unit tests plus live testing.
src/java/org/apache/sqoop/manager/DirectMySQLManager.java
Revision 2e8d63e5b45f0b74b0ccce9e9bff4a1f798bb6a8 New Change
[20] 101 lines
[+20] [+] public void exportTable(com.cloudera.sqoop.manager.ExportJobContext context)
102
    context.setConnManager(this);
102
    context.setConnManager(this);
103
    MySQLExportJob exportJob = new MySQLExportJob(context);
103
    MySQLExportJob exportJob = new MySQLExportJob(context);
104
    exportJob.runExport();
104
    exportJob.runExport();
105
  }
105
  }
106

    
   
106

   

    
   
107
  public void upsertTable(com.cloudera.sqoop.manager.ExportJobContext context)

    
   
108
      throws IOException, ExportException {

    
   
109
    throw new ExportException("MySQL direct connector does not support upsert" +

    
   
110
      " mode. Please use JDBC based connector (remove --direct parameter)");

    
   
111
  }

    
   
112

   
107
  @Override
113
  @Override
108
  public boolean supportsStagingForExport() {
114
  public boolean supportsStagingForExport() {
109
    return false;
115
    return false;
110
  }
116
  }
111
}
117
}
112

    
   
118

   
src/java/org/apache/sqoop/manager/MySQLManager.java
Revision a817aa41fee3385b6e8796cadd4c09319b0b6e68 New Change
 
src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java
Revision c8e17c236f272387fd14ef6d222cc0edb5fe59ab New Change
 
src/java/org/apache/sqoop/mapreduce/mysql/MySQLUpsertOutputFormat.java
New File
 
src/test/com/cloudera/sqoop/manager/JdbcMySQLExportTest.java
Revision f00cac4eb7c0600dc567717eff391909a831c6fb New Change
 
src/test/com/cloudera/sqoop/manager/ManualMySQLTests.java
New File
 
  1. src/java/org/apache/sqoop/manager/DirectMySQLManager.java: Loading...
  2. src/java/org/apache/sqoop/manager/MySQLManager.java: Loading...
  3. src/java/org/apache/sqoop/mapreduce/JdbcUpdateExportJob.java: Loading...
  4. src/java/org/apache/sqoop/mapreduce/mysql/MySQLUpsertOutputFormat.java: Loading...
  5. src/test/com/cloudera/sqoop/manager/JdbcMySQLExportTest.java: Loading...
  6. src/test/com/cloudera/sqoop/manager/ManualMySQLTests.java: Loading...