Review Board 1.7.22


SQOOP-1002 Sqoop2: Mapreduce local mode detection is not working correctly on Hadoop 2.x

Review Request #10687 - Created April 20, 2013 and submitted

Jarek Cecho
SQOOP-1002
Reviewers
Sqoop
sqoop-sqoop2
I've added framework detection. If it is set to "yarn" then we can't be running in mapreduce local mode.
Unit and integration tests seems to be passing. In addition I've verified the functionality on real Hadoop 2.x cluster.
submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
Revision 6607885ac609f49a0dd67440de64b6f18d4910f6 New Change
[20] 417 lines
[+20] [+] private Counters convertMapreduceCounters(org.apache.hadoop.mapred.Counters hadoopCounters) {
418
   * Detect MapReduce local mode.
418
   * Detect MapReduce local mode.
419
   *
419
   *
420
   * @return True if we're running in local mode
420
   * @return True if we're running in local mode
421
   */
421
   */
422
  private boolean isLocal() {
422
  private boolean isLocal() {

    
   
423
    // If framework is set to YARN, then we can't be running in local mode

    
   
424
    if("yarn".equals(globalConfiguration.get("mapreduce.framework.name"))) {

    
   
425
      return false;

    
   
426
    }

    
   
427

   

    
   
428
    // If job tracker address is "local" then we're running in local mode
423
    return "local".equals(globalConfiguration.get("mapreduce.jobtracker.address"))
429
    return "local".equals(globalConfiguration.get("mapreduce.jobtracker.address"))
424
        || "local".equals(globalConfiguration.get("mapred.job.tracker"));
430
        || "local".equals(globalConfiguration.get("mapred.job.tracker"));
425
  }
431
  }
426
}
432
}
  1. submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java: Loading...