Review Board 1.7.22


SQOOP-1049: Sqoop2: Record not imported if partition column value is NULL

Review Request #12451 - Created July 10, 2013 and submitted

Mengwei Ding
SQOOP-1049
Reviewers
Sqoop
jarcec
sqoop-sqoop2
commit c41d3e475700c4d2f97606b153d75a86ef543cf4
Author: Mengwei Ding <mengwei.ding@gmail.com>
Date:   Wed Jul 10 11:41:05 2013 -0700

    SQOOP-1049: Sqoop2: Record not imported if partition column value is NULL

:100644 100644 f3dee8e... 27db8af... M	common/src/main/java/org/apache/sqoop/model/FormUtils.java
:100644 100644 7792c57... 8d0c4ab... M	connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java
:100644 100644 ef27236... 0991b28... M	connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ImportTableForm.java
:100644 100644 44fc984... 0950e32... M	connector/connector-generic-jdbc/src/main/resources/generic-jdbc-connector-resources.properties
:100644 100644 7ecc900... a33dd6c... M	connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java
Have done a manual test, in which I successfully import a table with some null values in partition column.
common/src/main/java/org/apache/sqoop/model/FormUtils.java
Revision f3dee8e New Change
[20] 360 lines
[+20] [+] public static String toJson(Object configuration) {
361
            jsonForm.put(inputName, map);
361
            jsonForm.put(inputName, map);
362
          } else if(type == Integer.class) {
362
          } else if(type == Integer.class) {
363
            jsonForm.put(inputName, value);
363
            jsonForm.put(inputName, value);
364
          } else if(type.isEnum()) {
364
          } else if(type.isEnum()) {
365
            jsonForm.put(inputName, value.toString());
365
            jsonForm.put(inputName, value.toString());
366
          } else {
366
          } else if(type == Boolean.class) {

    
   
367
            jsonForm.put(inputName, value);

    
   
368
          }else {
367
            throw new SqoopException(ModelError.MODEL_004,
369
            throw new SqoopException(ModelError.MODEL_004,
368
              "Unsupported type " + type.getName() + " for input " + formName + "." + inputName);
370
              "Unsupported type " + type.getName() + " for input " + formName + "." + inputName);
369
          }
371
          }
370
        }
372
        }
371
      }
373
      }
[+20] [20] 76 lines
[+20] [+] public static void fillValues(String json, Object configuration) {
448
            inputField.set(formValue, map);
450
            inputField.set(formValue, map);
449
          } else if(type == Integer.class) {
451
          } else if(type == Integer.class) {
450
            inputField.set(formValue, ((Long)jsonInputs.get(inputName)).intValue());
452
            inputField.set(formValue, ((Long)jsonInputs.get(inputName)).intValue());
451
          } else if(type.isEnum()) {
453
          } else if(type.isEnum()) {
452
            inputField.set(formValue, Enum.valueOf((Class<? extends Enum>) inputField.getType(), (String) jsonInputs.get(inputName)));
454
            inputField.set(formValue, Enum.valueOf((Class<? extends Enum>) inputField.getType(), (String) jsonInputs.get(inputName)));
453
          } else {
455
          } else if(type == Boolean.class) {

    
   
456
            inputField.set(formValue, (Boolean) jsonInputs.get(inputName));

    
   
457
          }else {
454
            throw new SqoopException(ModelError.MODEL_004,
458
            throw new SqoopException(ModelError.MODEL_004,
455
              "Unsupported type " + type.getName() + " for input " + formName + "." + inputName);
459
              "Unsupported type " + type.getName() + " for input " + formName + "." + inputName);
456
          }
460
          }
457
        } catch (IllegalAccessException e) {
461
        } catch (IllegalAccessException e) {
458
          throw new SqoopException(ModelError.MODEL_005,
462
          throw new SqoopException(ModelError.MODEL_005,
459
            "Issue with field " + formName + "." + inputName, e);
463
            "Issue with field " + formName + "." + inputName, e);
460
        }
464
        }
461
      }
465
      }
462
    }
466
    }
463
  }
467
  }
464

    
   
468

   
465
}
469
}
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java
Revision 7792c57 New Change
 
connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ImportTableForm.java
Revision ef27236 New Change
 
connector/connector-generic-jdbc/src/main/resources/generic-jdbc-connector-resources.properties
Revision 44fc984 New Change
 
connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java
Revision 7ecc900 New Change
 
  1. common/src/main/java/org/apache/sqoop/model/FormUtils.java: Loading...
  2. connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportPartitioner.java: Loading...
  3. connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/configuration/ImportTableForm.java: Loading...
  4. connector/connector-generic-jdbc/src/main/resources/generic-jdbc-connector-resources.properties: Loading...
  5. connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestImportPartitioner.java: Loading...