Review Board 1.7.22


RollingFileSink need to be able to construct directory path based on escape sequence

Review Request #7131 - Created Sept. 17, 2012 and updated

Ted Malaska
trunk
FLUME-1295
Reviewers
Flume
flume-git
OK, first note this is still a work in progress, but I'm at the point where I need feedback.

Initial goal was to allow RollingFileSink to do Escape Sequence the same way as HDFSEventSink.

In order to reuse code, my change attempt to reuse bucketWriter and BucketWriterLinkedHashMap.

BucketWriter has been broken up into three files.

> AbstractBucketWriter : Contains all the rollover and escape sequence logic.
> BucketWriter: Contains the IO logic to HDFS
> FileBucketWriter: contains the IO logic to normal files

I also added a couple unit tests to RollingFileSink for new types of roll overs and escape sequences.


 
Total:
3
Open:
3
Resolved:
0
Dropped:
0
Status:
From:
Review request changed
Updated (Sept. 17, 2012, 10:34 p.m.)
Applied all of Brock's findings.
Posted (Dec. 4, 2012, 4:37 p.m.)
Echoing my comments from the JIRA. Personally what I think we should do with this patch is not modify HDFSEventSink but get all the "infrastructure" ready so we could modify HDFSEventSink once we are sure the new code is working correctly. The reason is HDFSEventSink is used by a ton of people and it's been a source of bugs in the past.