Review Board 1.7.22


HBASE-5869 Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb

Review Request #4926 - Created April 28, 2012 and updated

Michael Stack
HBASE-5869
Reviewers
hbase
jxiang
hbase-git
Convert two zk users to pb: distributed log splitting and regions in transition.

Refactored distributed log splitting so we only serialize/deserialize in one location.
Less changes needed to do same for regions in transition.

Moves serialization/deserialization out of the ZKAssign, ZKSplit and into
the classes themselves so can encapsulate how serialization is done into one place
(try to make the ZK* classes just deal in bytes -- about 90% done).

Moved classes used by various packages up to top level to minimize imports
that are across package (zookeeper into protobuf and/or into regionserver and/or
master packages, etc).

A src/main/java/org/apache/hadoop/hbase/DeserializationException.java
  New generic deserialization exception.
A src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java
D  src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java
  Moved under zookeeper package.
A src/main/java/org/apache/hadoop/hbase/HBaseException.java
  New base hbase exception as suggested by hbase-5796.  New DeserializationException
  inherits from this.
A src/main/java/org/apache/hadoop/hbase/RegionTransition.java
  State of a region in transition.  Top-level because used by a
  few top-level packages.  Encapsulates pb serialization/deserialization.
M src/main/java/org/apache/hadoop/hbase/ServerName.java
  Add method to deserialize a ServeName, etc.  Encapsulates pb'ing.
M src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java
  Counters used by distributed log splitting.
A SplitLogTask
   Class that encapsulates log splitting state.  Also encapsulates pb'ing.
M src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
  Implement code for state.  Added functions to go from code to state and vice
  versa.  Used serializing.
M src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java
  Remove unused imports.
D src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java
  Removed.  Replaced by RegionTransition moved to package top-level.
M src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
  Use new DeserializationException. Move to using new RegionTransition
  from RegionTransitionData class.  Pass deserialized class rather than
  byte array.  Remove duplicated code.
M src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  Use new ServerName parse method rather than ZKUtil one.
M src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
M src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java
  Redo to use new SplitLogTask and SplitLogCounter classes.
M src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
  expectPBMagicPrefix added
M src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
  Use new RegionTransition in place of RegionTransitionData.
M src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
  Define moved from ZKSplitLog to SplitLogManager.
M src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java
M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java
  Changed method name from getZNodeData to toByteArray to match how we've
  named it elsewhere. Use new DeserializationException
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java
  Use new RegionTransion class
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java
  Moved stuff that was in here up into SplitLogManager where better
  belongs.  Also moved serialization/deserialization up into the
  class itself: SplitLogTask.  Moved counters out to SplitLogCounter class.
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
  Moved deserialization of ServerName out of here and up into ServerName.
M src/main/protobuf/ZooKeeper.proto
  Add two new classes, RegionTransition and SplitLogTask.

 
Review request changed
Updated (May 1, 2012, 8:42 p.m.)
Same as original w/ a few fixes for tests that failed:

1. In distributed log tests, was failing to pick up the recovered.edits file because string passed included state of the split log task when what was wanted was servername only
Posted (May 1, 2012, 9:59 p.m.)

   

  
Usually this line is not the first in a file.
'log' seems redundant here.
What if an AtomicInteger counter is added in the future ?
  1. Open new JIRA.  This is just a move of existing code.
'date' -> 'data'
'An' -> 'A'
What would the first 64 bytes of data represent ?
Do we know that data.length >= 64 ?
  1. Let me fix toStringBinary so it deals w/ case where data is < 64 bytes.
    
    Regards what this represents, it could be anything.  Just saving our logs from being filled w/ binary.
Should e1 be included in the log ?
're-' before 'resubmit' is not necessary.
  1. Will add it.
    
    re- makes sense because this is retry inside exception handling.
Posted (May 1, 2012, 10:23 p.m.)

   

  
The port numbers don't match.