Review Board 1.7.22


BOOKKEEPER-160: bookie server needs to do compaction over entry log files to reclaim disk space

Review Request #3874 - Created Feb. 12, 2012 and updated

Sijie Guo
BOOKKEEPER-160
Reviewers
bookkeeper
bookkeeper-git
bookie server aggregates entries into entry log file. suppose there is lots of ledgers, each ledger has little messages. so a entry log file would contains messages from lots of different ledgers. if there is only one ledger not be deleted, the entry log file would not be removed, whose occupied disk space could not be reclaimed.

 
bookkeeper-server/conf/bk_server.conf
Revision d005d01 New Change
[20] 49 lines
[+20]
50

    
   
50

   
51
# Max file size of entry logger, in bytes
51
# Max file size of entry logger, in bytes
52
# A new entry log file will be created when the old one reaches the file size limitation
52
# A new entry log file will be created when the old one reaches the file size limitation
53
# logSizeLimit=2147483648
53
# logSizeLimit=2147483648
54

    
   
54

   

    
   
55
# Compaction Low Water Mark

    
   
56
# For those entry log files whose remaining size percentage reaches below

    
   
57
# low water mark will be compacted in a minor compaction.

    
   
58
# If both high water mark and low water mark are set to less than zero,

    
   
59
# the compaction is disabled.

    
   
60
# compactionLowWaterMark=0.0f

    
   
61

   

    
   
62
# Compaction High Water Mark

    
   
63
# For those entry log files whose remaining size percentage is still higher than

    
   
64
# high water mark, they will never be compacted. Those whose remaining size is between

    
   
65
# low water mark and high water mark will be compacted in a major compaction.

    
   
66
# If both high water mark and low water mark are set to less than zero,

    
   
67
# the compaction is disabled.

    
   
68
# compactionHighWaterMark=0.0f

    
   
69

   
55
# Max file size of journal file, in mega bytes
70
# Max file size of journal file, in mega bytes
56
# A new journal file will be created when the old one reaches the file size limitation
71
# A new journal file will be created when the old one reaches the file size limitation
57
#
72
#
58
# journalMaxSizeMB=2048
73
# journalMaxSizeMB=2048
59

    
   
74

   
[+20] [20] 80 lines
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
Revision d4ece94 New Change
 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java
Revision aca66e6 New Change
 
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java
New File
 
bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java
Revision 6bbe943 New Change
 
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java
New File
 
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java
Revision f661e90 New Change
 
  1. bookkeeper-server/conf/bk_server.conf: Loading...
  2. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java: Loading...
  3. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java: Loading...
  4. bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java: Loading...
  5. bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java: Loading...
  6. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java: Loading...
  7. bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java: Loading...