Review Board 1.7.22


SQOOP-863 Sqoop2: Introduce ProgressThread into Extractor and Loader

Review Request #9845 - Created March 11, 2013 and submitted

Jarek Cecho
SQOOP-863
Reviewers
Sqoop
sqoop-sqoop2
I've ported ProgressThread from Sqoop1 and plug it into SqoopMapper and SqoopReducer.
Unit and integration tests seems to be passing. I've also verified the functionality on real cluster.
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/ProgressRunnable.java
New File

    
   
1
/**

    
   
2
 * Licensed to the Apache Software Foundation (ASF) under one

    
   
3
 * or more contributor license agreements.  See the NOTICE file

    
   
4
 * distributed with this work for additional information

    
   
5
 * regarding copyright ownership.  The ASF licenses this file

    
   
6
 * to you under the Apache License, Version 2.0 (the

    
   
7
 * "License"); you may not use this file except in compliance

    
   
8
 * with the License.  You may obtain a copy of the License at

    
   
9
 *

    
   
10
 *     http://www.apache.org/licenses/LICENSE-2.0

    
   
11
 *

    
   
12
 * Unless required by applicable law or agreed to in writing, software

    
   
13
 * distributed under the License is distributed on an "AS IS" BASIS,

    
   
14
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

    
   
15
 * See the License for the specific language governing permissions and

    
   
16
 * limitations under the License.

    
   
17
 */

    
   
18
package org.apache.sqoop.job.mr;

    
   
19

   

    
   
20
import org.apache.commons.logging.Log;

    
   
21
import org.apache.commons.logging.LogFactory;

    
   
22
import org.apache.hadoop.conf.Configuration;

    
   
23
import org.apache.hadoop.mapreduce.TaskInputOutputContext;

    
   
24

   

    
   
25

   

    
   
26
/**

    
   
27
  * Runnable that will ping mapreduce context about progress.

    
   
28
  */

    
   
29
public class ProgressRunnable implements Runnable {

    
   
30

   

    
   
31
  public static final Log LOG = LogFactory.getLog(ProgressRunnable.class);

    
   
32

   

    
   
33
  /**

    
   
34
   * Context class that we should use for reporting progress.

    
   
35
   */

    
   
36
  private final TaskInputOutputContext context;

    
   
37

   

    
   
38
  public ProgressRunnable(final TaskInputOutputContext ctxt) {

    
   
39
    this.context = ctxt;

    
   
40
  }

    
   
41

   

    
   
42
  @Override

    
   
43
  public void run() {

    
   
44
    LOG.debug("Auto-progress thread reporting progress");

    
   
45
    this.context.progress();

    
   
46
  }

    
   
47
}
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java
Revision 2a823032cfd91fb8fd1a8aee3ffd9d80c2e3bae6 New Change
 
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopReducer.java
Revision d2361482771eea1a34b14c767952c5592f89c45a New Change
 
  1. execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/ProgressRunnable.java: Loading...
  2. execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java: Loading...
  3. execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopReducer.java: Loading...