Review Board 1.7.22


HBASE-4536 - Allow CF to retain deleted rows

Review Request #2178 - Created Oct. 4, 2011 and submitted

Lars Hofhansl
trunk
HBASE-4536
Reviewers
hbase
jgray, tedyu
hbase
HBase timerange Gets and Scans allow to do "timetravel" in HBase. I.e. look at the state of the data at any point in the past, provided the data is still around.
This did not work for deletes, however. Deletes would always mask all puts in the past.
This change adds a flag that can be on HColumnDescriptor to enable retention of deleted rows.
These rows are still subject to TTL and/or VERSIONS.

This changes the following:
1. There is a new flag on HColumnDescriptor enabling that behavior.
2. Allow gets/scans with a timerange to retrieve rows hidden by a delete marker, if the timerange does not include the delete marker.
3. Do not unconditionally collect all deleted rows during a compaction.
4. Allow a "raw" Scan, which retrieves all delete markers and deleted rows.

The change is small'ish, but the logic is intricate, so please review carefully.
All tests pass now.

Changes between revision 7 and 12

1 2 3 4 5 6 7 8 9 10 11 12
1 2 3 4 5 6 7 8 9 10 11 12

  1. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java: Loading...
  2. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java: Loading...
  3. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Attributes.java: Loading...
  4. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java: Loading...
  5. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java: Loading...
  6. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java: Loading...
  7. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java: Loading...
  8. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java: Loading...
  9. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java: Loading...
  10. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java: Loading...
  11. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java: Loading...
  12. http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb: Loading...
  13. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java: Loading...
  14. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java: Loading...
  15. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java: Loading...
  16. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java: Loading...
  17. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java: Loading...
  18. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java: Loading...
  19. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java: Loading...
  20. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java: Loading...
  21. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java: Loading...
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
Diff Revision 7 Diff Revision 12
[20] 85 lines
[+20] [+] public class HColumnDescriptor implements WritableComparable<HColumnDescriptor> {
86
  public static final String TTL = "TTL";
86
  public static final String TTL = "TTL";
87
  public static final String BLOOMFILTER = "BLOOMFILTER";
87
  public static final String BLOOMFILTER = "BLOOMFILTER";
88
  public static final String FOREVER = "FOREVER";
88
  public static final String FOREVER = "FOREVER";
89
  public static final String REPLICATION_SCOPE = "REPLICATION_SCOPE";
89
  public static final String REPLICATION_SCOPE = "REPLICATION_SCOPE";
90
  public static final String MIN_VERSIONS = "MIN_VERSIONS";
90
  public static final String MIN_VERSIONS = "MIN_VERSIONS";
91
  public static final String KEEP_DELETED_ROWS = "KEEP_DELETED_ROWS";
91
  public static final String KEEP_DELETED_CELLS = "KEEP_DELETED_CELLS";
92

    
   
92

   
93
  /**
93
  /**
94
   * Default compression type.
94
   * Default compression type.
95
   */
95
   */
96
  public static final String DEFAULT_COMPRESSION =
96
  public static final String DEFAULT_COMPRESSION =
[+20] [20] 58 lines
[+20] public class HColumnDescriptor implements WritableComparable<HColumnDescriptor> {
155
      DEFAULT_VALUES.put(COMPRESSION, DEFAULT_COMPRESSION);
155
      DEFAULT_VALUES.put(COMPRESSION, DEFAULT_COMPRESSION);
156
      DEFAULT_VALUES.put(TTL, String.valueOf(DEFAULT_TTL));
156
      DEFAULT_VALUES.put(TTL, String.valueOf(DEFAULT_TTL));
157
      DEFAULT_VALUES.put(BLOCKSIZE, String.valueOf(DEFAULT_BLOCKSIZE));
157
      DEFAULT_VALUES.put(BLOCKSIZE, String.valueOf(DEFAULT_BLOCKSIZE));
158
      DEFAULT_VALUES.put(HConstants.IN_MEMORY, String.valueOf(DEFAULT_IN_MEMORY));
158
      DEFAULT_VALUES.put(HConstants.IN_MEMORY, String.valueOf(DEFAULT_IN_MEMORY));
159
      DEFAULT_VALUES.put(BLOCKCACHE, String.valueOf(DEFAULT_BLOCKCACHE));
159
      DEFAULT_VALUES.put(BLOCKCACHE, String.valueOf(DEFAULT_BLOCKCACHE));
160
      DEFAULT_VALUES.put(KEEP_DELETED_ROWS, String.valueOf(DEFAULT_KEEP_DELETED));
160
      DEFAULT_VALUES.put(KEEP_DELETED_CELLS, String.valueOf(DEFAULT_KEEP_DELETED));
161
  }
161
  }
162

    
   
162

   
163
  // Column family name
163
  // Column family name
164
  private byte [] name;
164
  private byte [] name;
165

    
   
165

   
[+20] [20] 384 lines
[+20] [+] public boolean isInMemory() {
550
   */
550
   */
551
  public void setInMemory(boolean inMemory) {
551
  public void setInMemory(boolean inMemory) {
552
    setValue(HConstants.IN_MEMORY, Boolean.toString(inMemory));
552
    setValue(HConstants.IN_MEMORY, Boolean.toString(inMemory));
553
  }
553
  }
554

    
   
554

   
555
  public boolean getKeepDeletedRows() {
555
  public boolean getKeepDeletedCells() {
556
    String value = getValue(KEEP_DELETED_ROWS);
556
    String value = getValue(KEEP_DELETED_CELLS);
557
    if (value != null)
557
    if (value != null) {
558
      return Boolean.valueOf(value).booleanValue();
558
      return Boolean.valueOf(value).booleanValue();

    
   
559
    }
559
    return DEFAULT_KEEP_DELETED;
560
    return DEFAULT_KEEP_DELETED;
560
  }
561
  }
561

    
   
562

   
562
  /**
563
  /**
563
   * @param keepDeletedRows True if deleted rows should not be collected
564
   * @param keepDeletedRows True if deleted rows should not be collected
564
   * immediately.
565
   * immediately.
565
   */
566
   */
566
  public void setKeepDeletedRows(boolean keepDeletedRows) {
567
  public void setKeepDeletedRows(boolean keepDeletedRows) {
567
    setValue(KEEP_DELETED_ROWS, Boolean.toString(keepDeletedRows));
568
    setValue(KEEP_DELETED_CELLS, Boolean.toString(keepDeletedRows));
568
  }
569
  }
569

    
   
570

   
570
  /**
571
  /**
571
   * @return Time-to-live of cell contents, in seconds.
572
   * @return Time-to-live of cell contents, in seconds.
572
   */
573
   */
[+20] [20] 254 lines
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Attributes.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
Diff Revision 7 Diff Revision 12
 
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
Diff Revision 7 Diff Revision 12
 
  1. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java: Loading...
  2. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java: Loading...
  3. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Attributes.java: Loading...
  4. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java: Loading...
  5. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ColumnTracker.java: Loading...
  6. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java: Loading...
  7. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java: Loading...
  8. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java: Loading...
  9. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java: Loading...
  10. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java: Loading...
  11. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java: Loading...
  12. http://svn.apache.org/repos/asf/hbase/trunk/src/main/ruby/hbase/admin.rb: Loading...
  13. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java: Loading...
  14. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java: Loading...
  15. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java: Loading...
  16. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java: Loading...
  17. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java: Loading...
  18. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java: Loading...
  19. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java: Loading...
  20. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java: Loading...
  21. http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java: Loading...