RollingFileSink need to be able to construct directory path based on escape sequence
Review Request #7131 - Created Sept. 17, 2012 and updated
OK, first note this is still a work in progress, but I'm at the point where I need feedback. Initial goal was to allow RollingFileSink to do Escape Sequence the same way as HDFSEventSink. In order to reuse code, my change attempt to reuse bucketWriter and BucketWriterLinkedHashMap. BucketWriter has been broken up into three files. > AbstractBucketWriter : Contains all the rollover and escape sequence logic. > BucketWriter: Contains the IO logic to HDFS > FileBucketWriter: contains the IO logic to normal files I also added a couple unit tests to RollingFileSink for new types of roll overs and escape sequences.
|This is a hadoop issue and file systems which other sinks such as Rolling File sink won't have such an ...||Brock Noland||Sept. 17, 2012, 5:58 p.m.||Open|
|I think we should be using a File object to get the directory here.||Brock Noland||Sept. 17, 2012, 5:58 p.m.||Open|
|We should do an assertion or log something if mkdirs fails.||Brock Noland||Sept. 17, 2012, 5:58 p.m.||Open|