Review Board 1.7.22


HBASE-2195 - Support cyclic replication

Review Request #1730 - Created Sept. 6, 2011 and submitted

Lars Hofhansl
trunk
HBASE-2195
Reviewers
hbase
jdcryans, stack, tedyu
hbase
For Master <-> Master replication (as well as cycles > 2) a replication sink needs to keep track who is was the originator of a change and
(1) do not replicate this data back to the originator and
(2) pass that information on to the next link in a cycle > 2

In order to do that, HlogKeys read from the WAL are tagged with the Cluster UUID at the ReplicationSource before the logs are shipped to the Sink. The sink writes the Cluster UUID into the WAL log (in HlogKey, so only once per edit). If the sink is also source for yet another sink the ClusterUUID is retained in the HLogKeys read from the local WAL and passed on to the sink.
All tests pass.
New test: TestMasterReplication (which tests Master <-> Master, but no cycles > 2, yet), also passes.
Manually tested a cycle between 3 clusters.
Review request changed
Updated (Sept. 7, 2011, 11:19 p.m.)
Hopefully the last iteration.
Integrated Ted's optimization for ReplicationSource.removeNonReplicableEdits.
Better tests. Now tests Master - Master and a Cycle between 3 clusters.
Posted (Sept. 8, 2011, 2:45 a.m.)
The new TestMasterReplication passes
  1. Of course it does :)
I still don't see VersionedWritable.
  1. It does not really make sense to use VersionedWritable here. We can neither use readFields(...) not write(...) of VersionedWritable.
    
A short javadoc should be added, e.g.:

This class tests Master - Master replication and cyclic replication between 3 clusters.

Can be done at time of commit.