Review Board 1.7.22


SQOOP-674 Sqoop2: Exceptions in special map reduce threads can cause mapreduce job to freeze

Review Request #8622 - Created Dec. 16, 2012 and submitted

Jarek Cecho
SQOOP-674
Reviewers
Sqoop
hshreedharan
sqoop-sqoop2
I've changed unconditional free.acquire() to do-while loop that will periodically check reader status.
Unit tests are passing and I've tested it on real cluster.
execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java
Revision 38e2292781b10bab577f3aa8af47d2d0c9365e46 New Change
[20] 72 lines
[+20] [+] public class SqoopOutputFormatLoadExecutor {
73
   */
73
   */
74
  private class SqoopRecordWriter extends RecordWriter<Data, NullWritable> {
74
  private class SqoopRecordWriter extends RecordWriter<Data, NullWritable> {
75

    
   
75

   
76
    @Override
76
    @Override
77
    public void write(Data key, NullWritable value) throws InterruptedException {
77
    public void write(Data key, NullWritable value) throws InterruptedException {

    
   
78
      do {
78
      checkIfConsumerThrew();
79
        checkIfConsumerThrew();
79
      free.acquire();
80
      } while(!free.tryAcquire(5, TimeUnit.SECONDS));
80
      int type = key.getType();
81
      int type = key.getType();
81
      data.setContent(key.getContent(type), type);
82
      data.setContent(key.getContent(type), type);
82
      filled.release();
83
      filled.release();
83
    }
84
    }
84

    
   
85

   
[+20] [20] 137 lines
  1. execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java: Loading...