Review Board 1.7.22


SQOOP-773 Sqoop2: Batch execution support for client commands

Review Request #13181 - Created Aug. 1, 2013 and submitted

Abraham Elmahrek
SQOOP-773
Reviewers
Sqoop
hshreedharan, jarcec
sqoop-sqoop2
commit 080ced16578c1d95015ce3e99b4335beb465861a
Author: Abraham Elmahrek <abraham@elmahrek.com>
Date:   Tue Aug 13 14:14:06 2013 -0700

    SQOOP-773 Sqoop2: Batch execution support for client commands
    
    Separated options into two groups: fixed and dynamic options.
    Fixed options (IE: connector ID) come first and are used to select
    what options should be used in dynamic options. Dynamic options
    are automatically created based on forms selected from fixed options.
    The keys for these options take on the form "<prefix>-<form name>-<input-name>".

:100644 100644 980a908... a7e7e7d... M  shell/src/main/java/org/apache/sqoop/shell/CloneCommand.java
:100644 100644 856abaa... 2c2869c... M  shell/src/main/java/org/apache/sqoop/shell/CloneConnectionFunction.java
:100644 100644 3e23025... dd2eb2b... M  shell/src/main/java/org/apache/sqoop/shell/CloneJobFunction.java
:100644 100644 e62ce08... 9ad007b... M  shell/src/main/java/org/apache/sqoop/shell/CreateCommand.java
:100644 100644 5fbf0a3... 973fd53... M  shell/src/main/java/org/apache/sqoop/shell/CreateConnectionFunction.java
:100644 100644 6e4f04b... f0d4a6c... M  shell/src/main/java/org/apache/sqoop/shell/CreateJobFunction.java
:100644 100644 c123732... d79516d... M  shell/src/main/java/org/apache/sqoop/shell/DeleteConnectionFunction.java
:100644 100644 d4095b7... df9a2cc... M  shell/src/main/java/org/apache/sqoop/shell/DeleteJobFunction.java
:100644 100644 f119660... b5a54b9... M  shell/src/main/java/org/apache/sqoop/shell/DisableConnectionFunction.java
:100644 100644 a87e51f... 8cc4ea3... M  shell/src/main/java/org/apache/sqoop/shell/DisableJobFunction.java
:100644 100644 f782b16... 0c9f184... M  shell/src/main/java/org/apache/sqoop/shell/EnableConnectionFunction.java
:100644 100644 20c80dc... de5000b... M  shell/src/main/java/org/apache/sqoop/shell/EnableJobFunction.java
:100644 100644 1c43dce... f0153b8... M  shell/src/main/java/org/apache/sqoop/shell/SetOptionFunction.java
:100644 100644 abd9cea... 1aacd7c... M  shell/src/main/java/org/apache/sqoop/shell/SetServerFunction.java
:100644 100644 b55d5d1... f0720a0... M  shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
:100644 100644 97a4ab2... a9aba71... M  shell/src/main/java/org/apache/sqoop/shell/ShowConnectorFunction.java
:100644 100644 28497db... c71696c... M  shell/src/main/java/org/apache/sqoop/shell/ShowFrameworkFunction.java
:100644 100644 da4a4ff... 68bd3b3... M  shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
:100644 100644 5e3c3ff... d0b05e2... M  shell/src/main/java/org/apache/sqoop/shell/ShowOptionFunction.java
:100644 100644 ec97e63... 11bff86... M  shell/src/main/java/org/apache/sqoop/shell/ShowServerFunction.java
:100644 100644 a592a98... a571269... M  shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
:100644 100644 764b754... 5d6f97f... M  shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
:100644 100644 675a796... 371c867... M  shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
:100644 100644 f03e08f... 914454f... M  shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
:100644 100644 02148de... b8771c4... M  shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
:100644 100644 184892a... ebd4548... M  shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
:100644 100644 be0de8c... c61e9b9... M  shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
:100644 100644 698bca7... 65a454b... M  shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
:100644 100644 6c0e3c2... ae559cc... M  shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
:100644 100644 9262ccd... 24f31ea... M  shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
:100644 100644 c062fe6... 31d9af9... M  shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
:100644 100644 da1e0c5... 6f83b91... M  shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
:000000 100644 0000000... 6b6e858... A  shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
:000000 100644 0000000... cc63610... A  shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
:100644 100644 9bc0b93... bf0615e... M  shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
:000000 100644 0000000... 516790a... A  shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
:000000 100644 0000000... aa118e1... A  shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
Executed the following commands in a file:

create connection --cid 1 --name mysql-test --connector-connection-jdbcDriver com.mysql.jdbc.Driver --connector-connection-connectionString jdbc:mysql://hue.ent.cloudera.com/test --connector-connection-username root --connector-connection-password root
show connection
create job --xid 1 --type import --name mysql-import-job1 --connector-table-tableName test --connector-table-partitionColumn a --framework-output-storageType HDFS --framework-output-outputFormat TEXT_FILE --framework-output-outputDirectory /tmp/output
show job
start job --jid 1

Also executed the following commands in a file:
create connection --cid 1
create connection --cid 1

Verified erroneous commands stop the execution of the script. Verified statuses being returned will be visible on the screen when executing batch script.
shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java
Revision a592a9861f01d333ef72f1f96bdf038ab334ea9e New Change
[20] 26 lines
[+20]
27
import org.apache.sqoop.shell.utils.SubmissionDisplayer;
27
import org.apache.sqoop.shell.utils.SubmissionDisplayer;
28
import org.apache.sqoop.shell.utils.TableDisplayer;
28
import org.apache.sqoop.shell.utils.TableDisplayer;
29

    
   
29

   
30
import static org.apache.sqoop.shell.ShellEnvironment.*;
30
import static org.apache.sqoop.shell.ShellEnvironment.*;
31

    
   
31

   

    
   
32
@SuppressWarnings("serial")
32
public class ShowSubmissionFunction extends SqoopFunction {
33
public class ShowSubmissionFunction extends SqoopFunction {
33
  @SuppressWarnings("static-access")
34
  @SuppressWarnings("static-access")
34
  protected ShowSubmissionFunction() {
35
  protected ShowSubmissionFunction() {
35
    this.addOption(OptionBuilder
36
    this.addOption(OptionBuilder
36
        .withDescription(resourceString(Constants.RES_SHOW_PROMPT_DISPLAY_ALL_SUBMISSIONS))
37
        .withDescription(resourceString(Constants.RES_SHOW_PROMPT_DISPLAY_ALL_SUBMISSIONS))
[+20] [20] 4 lines
[+20] public class ShowSubmissionFunction extends SqoopFunction {
41
        .withLongOpt(Constants.OPT_JID)
42
        .withLongOpt(Constants.OPT_JID)
42
        .create(Constants.OPT_JID_CHAR));
43
        .create(Constants.OPT_JID_CHAR));
43
  }
44
  }
44

    
   
45

   
45
  @Override
46
  @Override
46
  public Object executeFunction(CommandLine line) {
47
  public Object executeInteractiveFunction(CommandLine line) {
47
    if (line.hasOption(Constants.OPT_DETAIL)) {
48
    if (line.hasOption(Constants.OPT_DETAIL)) {
48
      if (line.hasOption(Constants.OPT_JID)) {
49
      if (line.hasOption(Constants.OPT_JID)) {
49
        showSubmissions(getLong(line, Constants.OPT_JID));
50
        showSubmissions(getLong(line, Constants.OPT_JID));
50
      } else {
51
      } else {
51
        showSubmissions(null);
52
        showSubmissions(null);
[+20] [20] 7 lines
[+20] public Object executeFunction(CommandLine line) { public Object executeInteractiveFunction(CommandLine line) {
59
    }
60
    }
60

    
   
61

   
61
    return null;
62
    return null;
62
  }
63
  }
63

    
   
64

   

    
   
65
  public Object executeBatchFunction(CommandLine line) {

    
   
66
    return executeInteractiveFunction(line);

    
   
67
  }

    
   
68

   
64
  private void showSummary(Long jid) {
69
  private void showSummary(Long jid) {
65
    List<MSubmission> submissions;
70
    List<MSubmission> submissions;
66
    if (jid == null) {
71
    if (jid == null) {
67
      submissions = client.getSubmissions();
72
      submissions = client.getSubmissions();
68
    } else {
73
    } else {
[+20] [20] 37 lines
shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java
Revision 764b754180500a86501450c90b119813de6cf1f7 New Change
 
shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java
Revision 675a796d7919a3afadea2e14a88f7823d5dced47 New Change
 
shell/src/main/java/org/apache/sqoop/shell/StartCommand.java
Revision f03e08fe2178c2b044eef1bfbfa82c9235d8914d New Change
 
shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java
Revision 02148de497c15caa363c17a54202690a9026a68b New Change
 
shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java
Revision 184892a33b1b26391c7a7eaa8c79cd27f0178c39 New Change
 
shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java
Revision be0de8c29289a904cfc00361da7fbf9a6c8e7d1c New Change
 
shell/src/main/java/org/apache/sqoop/shell/StopCommand.java
Revision 698bca778d3aa09fa24372a6367986391f9e61d9 New Change
 
shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java
Revision 6c0e3c284b997561fc351a57e35ce2d6c77f6121 New Change
 
shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java
Revision 9262ccd085730c7101179157db9828ac709c267d New Change
 
shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java
Revision c062fe6781aebac15c4d59b5690772695d8a10a9 New Change
 
shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java
Revision da1e0c570fb43cea439549c0ab5268f67d55996d New Change
 
shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java
New File
 
shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java
New File
 
shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java
Revision 9bc0b93649cb2ae6c84228b52c04b60a36c13b18 New Change
 
shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java
New File
 
shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java
New File
 
  1. shell/src/main/java/org/apache/sqoop/shell/ShowSubmissionFunction.java: Loading...
  2. shell/src/main/java/org/apache/sqoop/shell/ShowVersionFunction.java: Loading...
  3. shell/src/main/java/org/apache/sqoop/shell/SqoopFunction.java: Loading...
  4. shell/src/main/java/org/apache/sqoop/shell/StartCommand.java: Loading...
  5. shell/src/main/java/org/apache/sqoop/shell/StartJobFunction.java: Loading...
  6. shell/src/main/java/org/apache/sqoop/shell/StatusCommand.java: Loading...
  7. shell/src/main/java/org/apache/sqoop/shell/StatusJobFunction.java: Loading...
  8. shell/src/main/java/org/apache/sqoop/shell/StopCommand.java: Loading...
  9. shell/src/main/java/org/apache/sqoop/shell/StopJobFunction.java: Loading...
  10. shell/src/main/java/org/apache/sqoop/shell/UpdateCommand.java: Loading...
  11. shell/src/main/java/org/apache/sqoop/shell/UpdateConnectionFunction.java: Loading...
  12. shell/src/main/java/org/apache/sqoop/shell/UpdateJobFunction.java: Loading...
  13. shell/src/main/java/org/apache/sqoop/shell/utils/ConnectionDynamicFormOptions.java: Loading...
  14. shell/src/main/java/org/apache/sqoop/shell/utils/DynamicFormOptions.java: Loading...
  15. shell/src/main/java/org/apache/sqoop/shell/utils/FormFiller.java: Loading...
  16. shell/src/main/java/org/apache/sqoop/shell/utils/FormOptions.java: Loading...
  17. shell/src/main/java/org/apache/sqoop/shell/utils/JobDynamicFormOptions.java: Loading...
This diff has been split across 2 pages: < 1 2