Review Board 1.7.22

HBASE-5869 Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb

Review Request #4926 - Created April 28, 2012 and updated

Michael Stack
Convert two zk users to pb: distributed log splitting and regions in transition.

Refactored distributed log splitting so we only serialize/deserialize in one location.
Less changes needed to do same for regions in transition.

Moves serialization/deserialization out of the ZKAssign, ZKSplit and into
the classes themselves so can encapsulate how serialization is done into one place
(try to make the ZK* classes just deal in bytes -- about 90% done).

Moved classes used by various packages up to top level to minimize imports
that are across package (zookeeper into protobuf and/or into regionserver and/or
master packages, etc).

A src/main/java/org/apache/hadoop/hbase/
  New generic deserialization exception.
A src/main/java/org/apache/hadoop/hbase/zookeeper/
D  src/main/java/org/apache/hadoop/hbase/
  Moved under zookeeper package.
A src/main/java/org/apache/hadoop/hbase/
  New base hbase exception as suggested by hbase-5796.  New DeserializationException
  inherits from this.
A src/main/java/org/apache/hadoop/hbase/
  State of a region in transition.  Top-level because used by a
  few top-level packages.  Encapsulates pb serialization/deserialization.
M src/main/java/org/apache/hadoop/hbase/
  Add method to deserialize a ServeName, etc.  Encapsulates pb'ing.
M src/main/java/org/apache/hadoop/hbase/
  Counters used by distributed log splitting.
A SplitLogTask
   Class that encapsulates log splitting state.  Also encapsulates pb'ing.
M src/main/java/org/apache/hadoop/hbase/executor/
  Implement code for state.  Added functions to go from code to state and vice
  versa.  Used serializing.
M src/main/java/org/apache/hadoop/hbase/executor/
  Remove unused imports.
D src/main/java/org/apache/hadoop/hbase/executor/
  Removed.  Replaced by RegionTransition moved to package top-level.
M src/main/java/org/apache/hadoop/hbase/master/
M src/main/java/org/apache/hadoop/hbase/master/
  Use new DeserializationException. Move to using new RegionTransition
  from RegionTransitionData class.  Pass deserialized class rather than
  byte array.  Remove duplicated code.
M src/main/java/org/apache/hadoop/hbase/master/
  Use new ServerName parse method rather than ZKUtil one.
M src/main/java/org/apache/hadoop/hbase/master/
M src/main/java/org/apache/hadoop/hbase/regionserver/
  Redo to use new SplitLogTask and SplitLogCounter classes.
M src/main/java/org/apache/hadoop/hbase/protobuf/
  expectPBMagicPrefix added
M src/main/java/org/apache/hadoop/hbase/regionserver/
  Use new RegionTransition in place of RegionTransitionData.
M src/main/java/org/apache/hadoop/hbase/regionserver/wal/
  Define moved from ZKSplitLog to SplitLogManager.
M src/main/java/org/apache/hadoop/hbase/zookeeper/
M src/main/java/org/apache/hadoop/hbase/zookeeper/
  Changed method name from getZNodeData to toByteArray to match how we've
  named it elsewhere. Use new DeserializationException
M src/main/java/org/apache/hadoop/hbase/zookeeper/
  Use new RegionTransion class
M src/main/java/org/apache/hadoop/hbase/zookeeper/
  Moved stuff that was in here up into SplitLogManager where better
  belongs.  Also moved serialization/deserialization up into the
  class itself: SplitLogTask.  Moved counters out to SplitLogCounter class.
M src/main/java/org/apache/hadoop/hbase/zookeeper/
  Moved deserialization of ServerName out of here and up into ServerName.
M src/main/protobuf/ZooKeeper.proto
  Add two new classes, RegionTransition and SplitLogTask.

Review request changed
Updated (May 1, 2012, 8:42 p.m.)
Same as original w/ a few fixes for tests that failed:

1. In distributed log tests, was failing to pick up the recovered.edits file because string passed included state of the split log task when what was wanted was servername only
Posted (May 1, 2012, 9:59 p.m.)


Usually this line is not the first in a file.
'log' seems redundant here.
What if an AtomicInteger counter is added in the future ?
  1. Open new JIRA.  This is just a move of existing code.
'date' -> 'data'
'An' -> 'A'
What would the first 64 bytes of data represent ?
Do we know that data.length >= 64 ?
  1. Let me fix toStringBinary so it deals w/ case where data is < 64 bytes.
    Regards what this represents, it could be anything.  Just saving our logs from being filled w/ binary.
Should e1 be included in the log ?
're-' before 'resubmit' is not necessary.
  1. Will add it.
    re- makes sense because this is retry inside exception handling.
Posted (May 1, 2012, 10:23 p.m.)


The port numbers don't match.