Review Board 1.7.22


HBASE-4071: Support minimum number of versions with TTL

Review Request #1582 - Created Aug. 18, 2011 and submitted

Lars Hofhansl
trunk
HBASE-4071
Reviewers
hbase
jgray, stack, tlipcon
hbase
Allow enforcing a minimum number of versions when TTL is enable for a store.
The GC logic for both versions and TTL is unified inside the ColumnTrackers.
Ran all tests. I get error (not failures) from two: TestDistributedLogSplitting and TestHTablePool. Both fail with or without my changes.
New tests: TestMinVersions.

Changes between revision 4 and 7

1 2 3 4 5 6 7
1 2 3 4 5 6 7

  1. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java: Loading...
  2. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java: Loading...
  3. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java: Loading...
  4. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java: Loading...
  5. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java: Loading...
  6. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java: Loading...
  7. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java: Loading...
  8. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java: Loading...
  9. http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb: Loading...
  10. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java: Loading...
  11. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java: Loading...
  12. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java: Loading...
  13. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java: Loading...
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
Diff Revision 4 Diff Revision 7
[20] 262 lines
[+20] [+] public class HColumnDescriptor implements WritableComparable<HColumnDescriptor> {
263
   */
263
   */
264
  public HColumnDescriptor(final byte [] familyName, final int maxVersions,
264
  public HColumnDescriptor(final byte [] familyName, final int maxVersions,
265
      final String compression, final boolean inMemory,
265
      final String compression, final boolean inMemory,
266
      final boolean blockCacheEnabled, final int blocksize,
266
      final boolean blockCacheEnabled, final int blocksize,
267
      final int timeToLive, final String bloomFilter, final int scope) {
267
      final int timeToLive, final String bloomFilter, final int scope) {
268
    this(familyName, DEFAULT_MIN_VERSIONS, maxVersions, compression, inMemory, blockCacheEnabled,
268
    this(familyName, DEFAULT_MIN_VERSIONS, maxVersions, compression, inMemory,
269
        blocksize, timeToLive, bloomFilter, scope);
269
        blockCacheEnabled, blocksize, timeToLive, bloomFilter, scope);
270
  }
270
  }
271

    
   
271

   
272
  /**
272
  /**
273
   * Constructor
273
   * Constructor
274
   * @param familyName Column family name. Must be 'printable' -- digit or
274
   * @param familyName Column family name. Must be 'printable' -- digit or
275
   * letter -- and may not contain a <code>:<code>
275
   * letter -- and may not contain a <code>:<code>
276
   * @param maxVersions Maximum number of versions to keep

   
277
   * @param minVersions Minimum number of versions to keep
276
   * @param minVersions Minimum number of versions to keep

    
   
277
   * @param maxVersions Maximum number of versions to keep
278
   * @param compression Compression type
278
   * @param compression Compression type
279
   * @param inMemory If true, column data should be kept in an HRegionServer's
279
   * @param inMemory If true, column data should be kept in an HRegionServer's
280
   * cache
280
   * cache
281
   * @param blockCacheEnabled If true, MapFile blocks should be cached
281
   * @param blockCacheEnabled If true, MapFile blocks should be cached
282
   * @param blocksize Block size to use when writing out storefiles.  Use
282
   * @param blocksize Block size to use when writing out storefiles.  Use
[+20] [20] 7 lines
[+20] public class HColumnDescriptor implements WritableComparable<HColumnDescriptor> {
290
   * @throws IllegalArgumentException if passed a family name that is made of
290
   * @throws IllegalArgumentException if passed a family name that is made of
291
   * other than 'word' characters: i.e. <code>[a-zA-Z_0-9]</code> or contains
291
   * other than 'word' characters: i.e. <code>[a-zA-Z_0-9]</code> or contains
292
   * a <code>:</code>
292
   * a <code>:</code>
293
   * @throws IllegalArgumentException if the number of versions is &lt;= 0
293
   * @throws IllegalArgumentException if the number of versions is &lt;= 0
294
   */
294
   */
295
  public HColumnDescriptor(final byte [] familyName, final int minVersions, final int maxVersions,
295
  public HColumnDescriptor(final byte [] familyName, final int minVersions,
296
      final String compression, final boolean inMemory,
296
      final int maxVersions, final String compression, final boolean inMemory,
297
      final boolean blockCacheEnabled, final int blocksize,
297
      final boolean blockCacheEnabled, final int blocksize,
298
      final int timeToLive, final String bloomFilter, final int scope) {
298
      final int timeToLive, final String bloomFilter, final int scope) {
299
    isLegalFamilyName(familyName);
299
    isLegalFamilyName(familyName);
300
    this.name = familyName;
300
    this.name = familyName;
301

    
   
301

   
302
    if (maxVersions <= 0) {
302
    if (maxVersions <= 0) {
303
      // TODO: Allow maxVersion of 0 to be the way you say "Keep all versions".
303
      // TODO: Allow maxVersion of 0 to be the way you say "Keep all versions".
304
      // Until there is support, consider 0 or < 0 -- a configuration error.
304
      // Until there is support, consider 0 or < 0 -- a configuration error.
305
      throw new IllegalArgumentException("Maximum versions must be positive");
305
      throw new IllegalArgumentException("Maximum versions must be positive");
306
    }
306
    }
307

    
   
307

   
308
    if (minVersions > 0) {
308
    if (minVersions > 0) {
309
      if (timeToLive == HConstants.FOREVER) {
309
      if (timeToLive == HConstants.FOREVER) {
310
        throw new IllegalArgumentException("Minimum versions requires TTL to set.");
310
        throw new IllegalArgumentException("Minimum versions requires TTL.");
311
      }
311
      }
312
      if (minVersions > maxVersions) {
312
      if (minVersions > maxVersions) {
313
        throw new IllegalArgumentException("Minimum versions must be <= maximum versions.");
313
        throw new IllegalArgumentException("Minimum versions must be <= "+

    
   
314
            "maximum versions.");
314
      }
315
      }
315
    }
316
    }
316

    
   
317

   
317
    setMaxVersions(maxVersions);
318
    setMaxVersions(maxVersions);
318
    setMinVersions(minVersions);
319
    setMinVersions(minVersions);
[+20] [20] 235 lines
[+20] [+] public int getTimeToLive() {
554
   */
555
   */
555
  public void setTimeToLive(int timeToLive) {
556
  public void setTimeToLive(int timeToLive) {
556
    setValue(TTL, Integer.toString(timeToLive));
557
    setValue(TTL, Integer.toString(timeToLive));
557
  }
558
  }
558

    
   
559

   

    
   
560
  /**

    
   
561
   * @return The minimum number of versions to keep.

    
   
562
   */
559
  public int getMinVersions() {
563
  public int getMinVersions() {
560
    String value = getValue(MIN_VERSIONS);
564
    String value = getValue(MIN_VERSIONS);
561
    return (value != null)? Integer.valueOf(value).intValue(): 0;
565
    return (value != null)? Integer.valueOf(value).intValue(): 0;
562
  }
566
  }
563

    
   
567

   

    
   
568
  /**

    
   
569
   * @param minVersions The minimum number of versions to keep.

    
   
570
   * (used when timeToLive is set)

    
   
571
   */
564
  public void setMinVersions(int minVersions) {
572
  public void setMinVersions(int minVersions) {
565
    setValue(MIN_VERSIONS, Integer.toString(minVersions));
573
    setValue(MIN_VERSIONS, Integer.toString(minVersions));
566
  }
574
  }
567

    
   
575

   
568
  /**
576
  /**
[+20] [20] 225 lines
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
Diff Revision 4 Diff Revision 7
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
Diff Revision 4 Diff Revision 7
 
  1. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java: Loading...
  2. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java: Loading...
  3. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java: Loading...
  4. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java: Loading...
  5. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java: Loading...
  6. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java: Loading...
  7. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java: Loading...
  8. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java: Loading...
  9. http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb: Loading...
  10. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java: Loading...
  11. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java: Loading...
  12. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java: Loading...
  13. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java: Loading...