Review Board 1.7.22


FLUME-1955: fileSuffix does not work with compressed streams

Review Request #10031 - Created March 19, 2013 and submitted

Mike Percy
FLUME-1955
Reviewers
Flume
flume-git
Minor patch to make fileSuffix work with compressed streams.
Added unit test, some manual testing.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
Revision 774f297 New Change
[20] 219 lines
[+20] [+] private void doOpen() throws IOException, InterruptedException {
220
      try {
220
      try {
221
        long counter = fileExtensionCounter.incrementAndGet();
221
        long counter = fileExtensionCounter.incrementAndGet();
222

    
   
222

   
223
        String fullFileName = fileName + "." + counter;
223
        String fullFileName = fileName + "." + counter;
224

    
   
224

   
225
        if (codeC == null && fileSuffix != null && fileSuffix.length() > 0) {
225
        if (fileSuffix != null && fileSuffix.length() > 0) {
226
          fullFileName += fileSuffix;
226
          fullFileName += fileSuffix;
227
        }
227
        } else if (codeC != null) {
228

    
   

   
229
        if(codeC != null) {

   
230
          fullFileName += codeC.getDefaultExtension();
228
          fullFileName += codeC.getDefaultExtension();
231
        }
229
        }
232

    
   
230

   
233
        bucketPath = filePath + DIRECTORY_DELIMITER + inUsePrefix
231
        bucketPath = filePath + DIRECTORY_DELIMITER + inUsePrefix
234
          + fullFileName + inUseSuffix;
232
          + fullFileName + inUseSuffix;
[+20] [20] 368 lines
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
Revision 741ac90 New Change
 
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java
Revision 99e787e New Change
 
  1. flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java: Loading...
  2. flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java: Loading...
  3. flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java: Loading...