Review Board 1.7.22


FLUME-1762. File Channel should recover automatically if the checkpoint is incomplete or bad by deleting the contents of the checkpoint directory

Review Request #8396 - Created Dec. 7, 2012 and submitted

Hari Shreedharan
FLUME-1762
Reviewers
Flume
flume-git
Added code to throw a BadCheckpointException, if we can recover from the situation by deleting all files in the checkpoint directory. In the log class, during startup if BadCheckpointException is caught, all files are deleted and replay is retried.
Added unit tests. Modified some existing unit tests to test for this change.
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/BadCheckpointException.java
New File

    
   
1
/*

    
   
2
 * Licensed to the Apache Software Foundation (ASF) under one

    
   
3
 * or more contributor license agreements.  See the NOTICE file

    
   
4
 * distributed with this work for additional information

    
   
5
 * regarding copyright ownership.  The ASF licenses this file

    
   
6
 * to you under the Apache License, Version 2.0 (the

    
   
7
 * "License"); you may not use this file except in compliance

    
   
8
 * with the License.  You may obtain a copy of the License at

    
   
9
 *

    
   
10
 *     http://www.apache.org/licenses/LICENSE-2.0

    
   
11
 *

    
   
12
 * Unless required by applicable law or agreed to in writing, software

    
   
13
 * distributed under the License is distributed on an "AS IS" BASIS,

    
   
14
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

    
   
15
 * See the License for the specific language governing permissions and

    
   
16
 * limitations under the License.

    
   
17
 */

    
   
18

   

    
   
19
package org.apache.flume.channel.file;

    
   
20

   

    
   
21
import org.apache.flume.FlumeException;

    
   
22

   

    
   
23
/**

    
   
24
 * Exception thrown when the checkpoint directory contains invalid data,

    
   
25
 * probably due to the channel stopping while the checkpoint was written.

    
   
26
 */

    
   
27
public class BadCheckpointException extends FlumeException{

    
   
28
  private static final long serialVersionUID = -5038652693746472779L;

    
   
29

   

    
   
30
  public BadCheckpointException(String msg) {

    
   
31
    super(msg);

    
   
32
  }

    
   
33
  public BadCheckpointException(String msg, Throwable t) {

    
   
34
    super(msg, t);

    
   
35
  }

    
   
36
}
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFactory.java
Revision 6c07152 New Change
 
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
Revision 5eaf8c2 New Change
 
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFileV2.java
Revision 8bbc081 New Change
 
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFileV3.java
Revision c24f89f New Change
 
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java
Revision 36553c5 New Change
 
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java
Revision 6d1cf51 New Change
 
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Serialization.java
Revision ef8cf72 New Change
 
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestEventQueueBackingStoreFactory.java
Revision b1a55be New Change
 
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java
Revision 3f90805 New Change
 
flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java
Revision 0173390 New Change
 
  1. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/BadCheckpointException.java: Loading...
  2. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFactory.java: Loading...
  3. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java: Loading...
  4. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFileV2.java: Loading...
  5. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFileV3.java: Loading...
  6. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java: Loading...
  7. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java: Loading...
  8. flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Serialization.java: Loading...
  9. flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestEventQueueBackingStoreFactory.java: Loading...
  10. flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java: Loading...
  11. flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java: Loading...