FLUME-1586. File Channel should support verifying integrity of individual events.
Review Request #10944 - Created May 6, 2013 and submitted
Patch to add a checksum to events and replace them with a noop event using a tool, if corrupt.
Added unit tests when corrupt and noop events are encountered. I will add tests for the tool as well soon. I have not yet tested the tool completely. This patch aims at gathering feedback on the approach.
|this. is not consistent with the above||Brock Noland||May 23, 2013, 10:01 p.m.||Open|
|Let's log the offset and file like the message above.||Brock Noland||May 23, 2013, 10:01 p.m.||Open|
|Curious why we have "" + ".old"?||Brock Noland||May 23, 2013, 10:01 p.m.||Open|
|This should be typed IllegalStateException no?||Brock Noland||May 23, 2013, 10:01 p.m.||Open|
|Curious as to what case would make eventPosition == -1?||Brock Noland||May 23, 2013, 10:01 p.m.||Open|
|There should be none. WIll remove the if.||Hari Shreedharan||May 23, 2013, 10:48 p.m.||Open|
Posted (May 23, 2013, 10:01 p.m.)
Hari, this is great! I have a few questions/comments below. I just did an initial pass. Also since we are making so many classes public should we add visibility annotations to those classes?
Posted (May 23, 2013, 10:48 p.m.)
This byte represents an event which was detected as corrupt, and then replaced by the tool. Basically the idea is that if this is seen the reads should simply ignore this event (log and ignore).
Will drop the "this." in the next patch.
Intellij craziness. Will fix.
There should be none. WIll remove the if.
Review request changed
Updated (June 13, 2013, 8:24 p.m.)
- changed from pending to submitted