HBASE-6573: Generic Three-Phase Commit framework
Review Request #6592 - Created Aug. 13, 2012 and updated
Distributed 3-Phase Commit framework that leverages ZK to coordinate between the commit coordinator (e.g. master) and the cohort members. The HBase 3PC is based on using 2PC on either side of the channel (both on the coordinator and cohort members) along with an operation specific error listener to keep track of operation failure. Implementation-wise, there are really just a couple of main classes: the coordinator, cohortMember and controllers for each. Currently, there controllers are hidden behind an interface with a ZooKeeper implementation. One of the nice things that is adds is the ability to pass an arbitrary exception from one of the cohort members back to the the coordinator and each of the other cohort members (which gets you meaningful error messages on the coordinator and, in the long view, back to the client). This abstracts out a lot of the work and complexity in snapshots (HBASE-6055) into a small, (relatively) easily tested change. 3PC is nice because "The basic observation is that in 2PC, while one site is in the “prepared to commit” state, the other may be in either the “commit” or the “abort” state. From this analysis, they developed 3PC to avoid such states and it is thus resilient to such failures." - Wikipedia (http://en.wikipedia.org/wiki/Three-phase_commit_protocol).
Ran added test 10x locally without failure.