Review Board 1.7.22


Adds an abort validation handler which terminates the processing of import/export jobs.

Review Request #10252 - Created April 3, 2013 and updated

Seetharam Venkatesh
SQOOP-916
Reviewers
Sqoop
jarcec
sqoop-trunk
Validation of the data copied, either import or export, is implemented today by comparing the row counts from the source and the target post copy. This issue adds a handler that aborts the processing notifying the user of the anomaly.

Makes this the default implementation.
Yes. Unit tests are added to test the new functionality.
src/docs/user/validation.txt
Revision f28b420 New Change
[20] 87 lines
[+20]
88
.Validation Failure Handler
88
.Validation Failure Handler
89
 Property:         validation-failurehandler
89
 Property:         validation-failurehandler
90
 Description:      Responsible for handling failures, must implement
90
 Description:      Responsible for handling failures, must implement
91
                   org.apache.sqoop.validation.ValidationFailureHandler
91
                   org.apache.sqoop.validation.ValidationFailureHandler
92
 Supported values: The value has to be a fully qualified class name.
92
 Supported values: The value has to be a fully qualified class name.
93
 Default value:    org.apache.sqoop.validation.LogOnFailureHandler
93
 Default value:    org.apache.sqoop.validation.AbortOnFailureHandler
94

    
   
94

   
95

    
   
95

   
96
Limitations
96
Limitations
97
~~~~~~~~~~~
97
~~~~~~~~~~~
98

    
   
98

   
[+20] [20] 31 lines
[+20]
130
$ sqoop import --connect jdbc:mysql://db.foo.com/corp --table EMPLOYEES \
130
$ sqoop import --connect jdbc:mysql://db.foo.com/corp --table EMPLOYEES \
131
    --validate --validator org.apache.sqoop.validation.RowCountValidator \
131
    --validate --validator org.apache.sqoop.validation.RowCountValidator \
132
    --validation-threshold \
132
    --validation-threshold \
133
          org.apache.sqoop.validation.AbsoluteValidationThreshold \
133
          org.apache.sqoop.validation.AbsoluteValidationThreshold \
134
    --validation-failurehandler \
134
    --validation-failurehandler \
135
          org.apache.sqoop.validation.LogOnFailureHandler
135
          org.apache.sqoop.validation.AbortOnFailureHandler
136
----
136
----
src/java/org/apache/sqoop/SqoopOptions.java
Revision 906542b New Change
 
src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
New File
 
src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
Revision 3ded6ad New Change
 
src/java/org/apache/sqoop/validation/RowCountValidator.java
Revision 2896192 New Change
 
src/java/org/apache/sqoop/validation/ValidationException.java
Revision 0c10241 New Change
 
src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
New File
 
src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
New File
 
src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
Revision a53e281 New Change
 
  1. src/docs/user/validation.txt: Loading...
  2. src/java/org/apache/sqoop/SqoopOptions.java: Loading...
  3. src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java: Loading...
  4. src/java/org/apache/sqoop/validation/LogOnFailureHandler.java: Loading...
  5. src/java/org/apache/sqoop/validation/RowCountValidator.java: Loading...
  6. src/java/org/apache/sqoop/validation/ValidationException.java: Loading...
  7. src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java: Loading...
  8. src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java: Loading...
  9. src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java: Loading...