Review Board 1.7.22


SQOOP-678 Sqoop2: Add counters handling to map reduce submission engine

Review Request #8782 - Created Dec. 29, 2012 and submitted

Jarek Cecho
SQOOP-678
Reviewers
Sqoop
sqoop-sqoop2
I've added counter propagation function to mapreduce submission engine.
Tested on real cluster.
core/src/main/java/org/apache/sqoop/framework/FrameworkManager.java
Revision 40e1f0b128a9b70067f731a19c71b65377a9a74f New Change
[20] 563 lines
[+20] [+] private void update(MSubmission submission) {
564
    String externalLink = submissionEngine.externalLink(externalId);
564
    String externalLink = submissionEngine.externalLink(externalId);
565

    
   
565

   
566
    if(newStatus.isRunning()) {
566
    if(newStatus.isRunning()) {
567
      progress = submissionEngine.progress(externalId);
567
      progress = submissionEngine.progress(externalId);
568
    } else {
568
    } else {
569
      counters = submissionEngine.stats(externalId);
569
      counters = submissionEngine.counters(externalId);
570
    }
570
    }
571

    
   
571

   
572
    submission.setStatus(newStatus);
572
    submission.setStatus(newStatus);
573
    submission.setProgress(progress);
573
    submission.setProgress(progress);
574
    submission.setCounters(counters);
574
    submission.setCounters(counters);
[+20] [20] 59 lines
core/src/main/java/org/apache/sqoop/framework/SubmissionEngine.java
Revision 71e4ec97af42fff85b0d749e4b69b9cbc553016e New Change
 
submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java
Revision 5c57758f92032ef802a07a1af7d70990d8e12a66 New Change
 
  1. core/src/main/java/org/apache/sqoop/framework/FrameworkManager.java: Loading...
  2. core/src/main/java/org/apache/sqoop/framework/SubmissionEngine.java: Loading...
  3. submission/mapreduce/src/main/java/org/apache/sqoop/submission/mapreduce/MapreduceSubmissionEngine.java: Loading...