Review Board 1.7.22


Fix for SQOOP-937

Review Request #9909 - Created March 13, 2013 and updated

Venkat Ranganathan
Reviewers
Sqoop
jarcec
sqoop-trunk
SQOOP generates an ORM file that represents a record in a table for a
given connector implememntation.  The generated class has methods to
read field values off ResultSet, set bind values in PreparedStatements,
handle LOB objects in a DB specific way (that the connector represents),
parse input fields etc.

This file is then compiled and archived using jar and used with the SQOOP job.

This ORM instance is generated in all cases, and used to make sure that
the data is read and processed in a more or less uniform way.

Unfortunately, this generated class is not used by a class of connectors which
manage the reading and processing of records themselves.   In essence, the
whole ORM class is unusable in these instances and is simply ignored.  These
are typically "direct" connectors which use a DB specific highspeed path.

The generation of the ORM class in these cases causes confusion to users.

This patch tries to solve this by

  1)  Providing a capability for the connection managers to declare that
they don't depend on the ORM jar file.
  2) Generating a dummy ORM jar file with explicit message during generation
so that
     a)  users are aware that the class generated is a dummy one
     b)  there is a record that this jar file was generated explicitly
     c)  we don't have to change a whole lot of the codebase to disable
         the generaration and loading of the jar file.


This patch also adds one test.
Added one tests.  All unit tests and check style tests passed with no new checkstyle issues
src/java/org/apache/sqoop/manager/ConnManager.java
Revision 1b32dc9 New Change
[20] 636 lines
[+20] [+] public String datetimeToQueryString(String datetime, int columnType) {
637
   * acceptable.
637
   * acceptable.
638
   */
638
   */
639
  public String getInputBoundsQuery(String splitByCol, String sanitizedQuery) {
639
  public String getInputBoundsQuery(String splitByCol, String sanitizedQuery) {
640
      return null;
640
      return null;
641
  }
641
  }

    
   
642

   

    
   
643
  /**

    
   
644
   * This method allows the ConnManager to override the generation of ORM

    
   
645
   * classes if the SQOOP generated classes are not used by it.

    
   
646
   * A return value of false from this method means that the SQOOP ORM

    
   
647
   * classes are needed to use with the connector.

    
   
648
   * A return value of true indicates that the connection manager does not

    
   
649
   * use the SQOOP ORM classes.   For example, in the Direct mode of some of

    
   
650
   * the connectors, the text files are directly processed by DB specific

    
   
651
   * facilities without even being passed through the SQOOP process and

    
   
652
   * in those circumstances, it makes sense to disable the ORM generation.

    
   
653
   */

    
   
654
  public boolean isORMFacilitySelfManaged() {

    
   
655
    return false;

    
   
656
  }
642
}
657
}
643

    
   
658

   
src/java/org/apache/sqoop/manager/DirectNetezzaManager.java
Revision 0a1e605 New Change
 
src/java/org/apache/sqoop/manager/ExportJobContext.java
Revision 5699e2f New Change
 
src/java/org/apache/sqoop/manager/ImportJobContext.java
Revision 09a7abe New Change
 
src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
Revision ff84974 New Change
 
src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
Revision f766532 New Change
 
src/java/org/apache/sqoop/mapreduce/JobBase.java
Revision 4e7723f New Change
 
src/java/org/apache/sqoop/orm/ClassWriter.java
Revision 982e444 New Change
 
src/java/org/apache/sqoop/tool/CodeGenTool.java
Revision 8a4aa42 New Change
 
src/test/com/cloudera/sqoop/orm/TestClassWriter.java
Revision 3b77571 New Change
 
  1. src/java/org/apache/sqoop/manager/ConnManager.java: Loading...
  2. src/java/org/apache/sqoop/manager/DirectNetezzaManager.java: Loading...
  3. src/java/org/apache/sqoop/manager/ExportJobContext.java: Loading...
  4. src/java/org/apache/sqoop/manager/ImportJobContext.java: Loading...
  5. src/java/org/apache/sqoop/mapreduce/ExportJobBase.java: Loading...
  6. src/java/org/apache/sqoop/mapreduce/ImportJobBase.java: Loading...
  7. src/java/org/apache/sqoop/mapreduce/JobBase.java: Loading...
  8. src/java/org/apache/sqoop/orm/ClassWriter.java: Loading...
  9. src/java/org/apache/sqoop/tool/CodeGenTool.java: Loading...
  10. src/test/com/cloudera/sqoop/orm/TestClassWriter.java: Loading...