Review Board 1.7.22


HDFS Sink should check if file is closed and retry if it is not.

Review Request #11583 - Created May 31, 2013 and updated

Ted Malaska
trunk
2007
Reviewers
Flume
flume-git
We can use the new API added in HDFS-4525. We will need to use reflection though, so we can run against a version of HDFS which does not have this API.


 
Total:
17
Open:
6
Resolved:
11
Dropped:
0
Status:
From:
Description From Last Updated Status
Why are you using wait() here? A sleep should be enough. I don't think this needs to be in a ... Hari Shreedharan June 7, 2013, 6:09 p.m. Open
The loop already checks that you try invoking close only a fixed number of times. Also, currently we don't throw ... Hari Shreedharan June 7, 2013, 6:09 p.m. Open
Is this new method really required? Does not look like it is. Hari Shreedharan June 7, 2013, 6:09 p.m. Open
I don't see this class anywhere in the patch, maybe missed a git add? Hari Shreedharan June 7, 2013, 6:09 p.m. Open
This parameter comes from the Flume configuration, not the HDFS configuration. So this will always be 30000. You probably need ... Hari Shreedharan July 1, 2013, 9:47 p.m. Open
This test seems to fail as a result of the last change. Could you please fix this one? Hari Shreedharan July 11, 2013, 2:52 a.m. Open
Review request changed
Updated (July 11, 2013, 1:28 a.m.)
Hari is the man
Ship it!
Posted (July 11, 2013, 2:19 a.m.)
Ship It!
Posted (July 11, 2013, 2:52 a.m.)

   

  
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java (Diff revision 6)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
This test seems to fail as a result of the last change. Could you please fix this one?