Review Board 1.7.22


HBASE-5869 Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb

Review Request #4926 - Created April 28, 2012 and updated

Michael Stack
HBASE-5869
Reviewers
hbase
jxiang
hbase-git
Convert two zk users to pb: distributed log splitting and regions in transition.

Refactored distributed log splitting so we only serialize/deserialize in one location.
Less changes needed to do same for regions in transition.

Moves serialization/deserialization out of the ZKAssign, ZKSplit and into
the classes themselves so can encapsulate how serialization is done into one place
(try to make the ZK* classes just deal in bytes -- about 90% done).

Moved classes used by various packages up to top level to minimize imports
that are across package (zookeeper into protobuf and/or into regionserver and/or
master packages, etc).

A src/main/java/org/apache/hadoop/hbase/DeserializationException.java
  New generic deserialization exception.
A src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java
D  src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java
  Moved under zookeeper package.
A src/main/java/org/apache/hadoop/hbase/HBaseException.java
  New base hbase exception as suggested by hbase-5796.  New DeserializationException
  inherits from this.
A src/main/java/org/apache/hadoop/hbase/RegionTransition.java
  State of a region in transition.  Top-level because used by a
  few top-level packages.  Encapsulates pb serialization/deserialization.
M src/main/java/org/apache/hadoop/hbase/ServerName.java
  Add method to deserialize a ServeName, etc.  Encapsulates pb'ing.
M src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java
  Counters used by distributed log splitting.
A SplitLogTask
   Class that encapsulates log splitting state.  Also encapsulates pb'ing.
M src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
  Implement code for state.  Added functions to go from code to state and vice
  versa.  Used serializing.
M src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
  Remove unused imports.
D src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java
  Removed.  Replaced by RegionTransition moved to package top-level.
M src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
  Use new DeserializationException. Move to using new RegionTransition
  from RegionTransitionData class.  Pass deserialized class rather than
  byte array.  Remove duplicated code.
M src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  Use new ServerName parse method rather than ZKUtil one.
M src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
M src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java
  Redo to use new SplitLogTask and SplitLogCounter classes.
M src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
  expectPBMagicPrefix added
M src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
  Use new RegionTransition in place of RegionTransitionData.
M src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
  Define moved from ZKSplitLog to SplitLogManager.
M src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java
M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
  Changed method name from getZNodeData to toByteArray to match how we've
  named it elsewhere. Use new DeserializationException
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
  Use new RegionTransion class
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java
  Moved stuff that was in here up into SplitLogManager where better
  belongs.  Also moved serialization/deserialization up into the
  class itself: SplitLogTask.  Moved counters out to SplitLogCounter class.
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
  Moved deserialization of ServerName out of here and up into ServerName.
M src/main/protobuf/ZooKeeper.proto
  Add two new classes, RegionTransition and SplitLogTask.

 
src/main/java/org/apache/hadoop/hbase/RegionTransition.java
Diff Revision 1 Diff Revision 3
[20] 105 lines
[+20] [+] public static RegionTransition createRegionTransition(final EventType type,
106
      org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.ServerName.newBuilder().
106
      org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.ServerName.newBuilder().
107
        setHostName(sn.getHostname()).setPort(sn.getPort()).setStartCode(sn.getStartcode()).build();
107
        setHostName(sn.getHostname()).setPort(sn.getPort()).setStartCode(sn.getStartcode()).build();
108
    ZooKeeperProtos.RegionTransition.Builder builder = ZooKeeperProtos.RegionTransition.newBuilder().
108
    ZooKeeperProtos.RegionTransition.Builder builder = ZooKeeperProtos.RegionTransition.newBuilder().
109
      setEventTypeCode(type.getCode()).setRegionName(ByteString.copyFrom(regionName)).
109
      setEventTypeCode(type.getCode()).setRegionName(ByteString.copyFrom(regionName)).
110
        setOriginServerName(pbsn);
110
        setOriginServerName(pbsn);

    
   
111
    builder.setCreateTime(System.currentTimeMillis());
111
    if (payload != null) builder.setPayload(ByteString.copyFrom(payload));
112
    if (payload != null) builder.setPayload(ByteString.copyFrom(payload));
112
    return new RegionTransition(builder.build());
113
    return new RegionTransition(builder.build());
113
  }
114
  }
114

    
   
115

   
115
  /**
116
  /**
[+20] [20] 25 lines
src/main/java/org/apache/hadoop/hbase/SplitLogTask.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
Diff Revision 1 Diff Revision 3
 
src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/TestSerialization.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
Diff Revision 1 Diff Revision 3
 
src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java
Diff Revision 1 Diff Revision 3
 
  1. src/main/java/org/apache/hadoop/hbase/RegionTransition.java: Loading...
  2. src/main/java/org/apache/hadoop/hbase/SplitLogTask.java: Loading...
  3. src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java: Loading...
  4. src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java: Loading...
  5. src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java: Loading...
  6. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java: Loading...
  7. src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java: Loading...
  8. src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java: Loading...
  9. src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java: Loading...
  10. src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java: Loading...
  11. src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java: Loading...
  12. src/test/java/org/apache/hadoop/hbase/TestSerialization.java: Loading...
  13. src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java: Loading...
  14. src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java: Loading...
  15. src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java: Loading...
  16. src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java: Loading...
  17. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java: Loading...
  18. src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java: Loading...
  19. src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java: Loading...