Review Board 1.7.22


SQOOP-746 Sqoop2: Mapreduce job should have running with speculative execution disabled

Review Request #8436 - Created Dec. 9, 2012 and submitted

Jarek Cecho
SQOOP-746
Reviewers
Sqoop
sqoop-sqoop2
I've disabled speculative execution in mapreduce submission engine.

 
submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
Revision 8f7864eba448774b7313a283ea3f617f58ad749f New Change
[20] 171 lines
[+20] [+] public boolean submit(SubmissionRequest generalRequest) {
172
    // Set up notification URL if it's available
172
    // Set up notification URL if it's available
173
    if(request.getNotificationUrl() != null) {
173
    if(request.getNotificationUrl() != null) {
174
      configuration.set("job.end.notification.url", request.getNotificationUrl());
174
      configuration.set("job.end.notification.url", request.getNotificationUrl());
175
    }
175
    }
176

    
   
176

   

    
   
177
    // Turn off speculative execution

    
   
178
    configuration.setBoolean("mapred.map.tasks.speculative.execution", false);

    
   
179
    configuration.setBoolean("mapred.reduce.tasks.speculative.execution", false);

    
   
180

   
177
    // Promote all required jars to the job
181
    // Promote all required jars to the job
178
    StringBuilder sb = new StringBuilder();
182
    StringBuilder sb = new StringBuilder();
179
    boolean first = true;
183
    boolean first = true;
180
    for(String jar : request.getJars()) {
184
    for(String jar : request.getJars()) {
181
      if(first) {
185
      if(first) {
[+20] [20] 162 lines
  1. submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java: Loading...