Review Board 1.7.22


FLUME-1685: ExecSource shouldn't die if the channel is full

Review Request #9003 - Created Jan. 17, 2013 and updated

Steve Hoffman
FLUME-1685
Reviewers
Flume
flume-git
Rather than kill the source outright when a channel fills, the channel should throw away the event (since there is nowhere to put it).  A basic back-off mechanism is also implemented so retries get spaced out the longer the channel remains full up to a configurable limit.

Patches are attached to FLUME-1685 including a backport to the 1.2 branch (which I use with Cloudera's cdh3u5 distribution)
A unit test was created to test the full channel condition.
This patch has been running in our production environment for 2 months and has successfully recovered several times from full channel events.
.gitignore
Revision 08c33a0 New Change
[20] 12 lines
[+20]
13
*/bin
13
*/bin
14
target
14
target
15
derby.log
15
derby.log
16
.idea
16
.idea
17
*.iml
17
*.iml

    
   
18
*.iws

    
   
19
*.ipr
18
nb-configuration.xml
20
nb-configuration.xml
flume-ng-core/pom.xml
Revision d01baca New Change
 
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java
Revision 46f672f New Change
 
flume-ng-core/src/main/java/org/apache/flume/source/ExecSourceConfigurationConstants.java
Revision 0ba0508 New Change
 
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java
Revision 8bcf320 New Change
 
  1. .gitignore: Loading...
  2. flume-ng-core/pom.xml: Loading...
  3. flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java: Loading...
  4. flume-ng-core/src/main/java/org/apache/flume/source/ExecSourceConfigurationConstants.java: Loading...
  5. flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java: Loading...