HDFS Sink should check if file is closed and retry if it is not.
Review Request #11583 - Created May 31, 2013 and updated
We can use the new API added in HDFS-4525. We will need to use reflection though, so we can run against a version of HDFS which does not have this API.
|Why are you using wait() here? A sleep should be enough. I don't think this needs to be in a ...||Hari Shreedharan||June 7, 2013, 6:09 p.m.||Open|
|The loop already checks that you try invoking close only a fixed number of times. Also, currently we don't throw ...||Hari Shreedharan||June 7, 2013, 6:09 p.m.||Open|
|Is this new method really required? Does not look like it is.||Hari Shreedharan||June 7, 2013, 6:09 p.m.||Open|
|I don't see this class anywhere in the patch, maybe missed a git add?||Hari Shreedharan||June 7, 2013, 6:09 p.m.||Open|
|This parameter comes from the Flume configuration, not the HDFS configuration. So this will always be 30000. You probably need ...||Hari Shreedharan||July 1, 2013, 9:47 p.m.||Open|
|This test seems to fail as a result of the last change. Could you please fix this one?||Hari Shreedharan||July 11, 2013, 2:52 a.m.||Open|