Review Board 1.7.22


FLUME-1893. File Channel could miss possible checkpoint corruption

Review Request #9266 - Created Feb. 2, 2013 and submitted

Hari Shreedharan
FLUME-1893
Reviewers
Flume
flume-git
Made sure the comparisons made are long comparisons and we don't miss the Most significant 4 bytes.
Added unit tests for the cases being fixed. All current tests pass.
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
Revision 7f35301 New Change
[20] 80 lines
[+20]
81
    }
81
    }
82
    mappedBuffer = checkpointFileHandle.getChannel().map(MapMode.READ_WRITE, 0,
82
    mappedBuffer = checkpointFileHandle.getChannel().map(MapMode.READ_WRITE, 0,
83
        checkpointFile.length());
83
        checkpointFile.length());
84
    elementsBuffer = mappedBuffer.asLongBuffer();
84
    elementsBuffer = mappedBuffer.asLongBuffer();
85

    
   
85

   
86
    int version = (int) elementsBuffer.get(INDEX_VERSION);
86
    long version = elementsBuffer.get(INDEX_VERSION);
87
    if(version != getVersion()) {
87
    if(version != (long) getVersion()) {
88
      throw new BadCheckpointException("Invalid version: " + version + " " +
88
      throw new BadCheckpointException("Invalid version: " + version + " " +
89
              name + ", expected " + getVersion());
89
              name + ", expected " + getVersion());
90
    }
90
    }
91
    long checkpointComplete =
91
    long checkpointComplete = elementsBuffer.get(INDEX_CHECKPOINT_MARKER);
92
        (int) elementsBuffer.get(INDEX_CHECKPOINT_MARKER);
92
    if(checkpointComplete != (long) CHECKPOINT_COMPLETE) {
93
    if(checkpointComplete != CHECKPOINT_COMPLETE) {

   
94
      throw new BadCheckpointException("Checkpoint was not completed correctly,"
93
      throw new BadCheckpointException("Checkpoint was not completed correctly,"
95
              + " probably because the agent stopped while the channel was"
94
              + " probably because the agent stopped while the channel was"
96
              + " checkpointing.");
95
              + " checkpointing.");
97
    }
96
    }
98
  }
97
  }
[+20] [20] 192 lines
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
Revision d98209b New Change
 
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java
Revision 7da8c49 New Change
 
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
Revision 170dc72 New Change
 
  1. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java: Loading...
  2. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java: Loading...
  3. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java: Loading...
  4. flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java: Loading...