Review Board 1.7.22


FLUME-1660 Close "idle" hdfs handles

Review Request #7659 - Created Oct. 19, 2012 and submitted

Juhani Connolly
FLUME-1660
Reviewers
Flume
mpercy
flume-git
Added lastWrite to BucketWriter to verify when it was last updated

Added a thread to HDFSEventSink which verifies the last update of each active bucketWriter and closes them after the configurable timeout hdfs.closeIdleTimeout has passed.
Local machine testing was performed and the correct closing of files was confirmed, as well as the correct behavior of the configuration setting including disabling the feature(by using the default value for hdfs.closeIdleTimeout of 0)


There is one unrelated test failure which I'm not sure of(if anyone knows what's causing this, please let me know)

Failed tests:   testInOut(org.apache.flume.test.agent.TestFileChannel): Expected FILE_ROLL sink's dir to have only 1 child, but found 0 children. expected:<1> but was:<0>

All other tests pass.
Total:
4
Open:
0
Resolved:
4
Dropped:
0
Status:
From:
Review request changed
Updated (Nov. 19, 2012, 8:41 a.m.)
  • changed from pending to submitted