Review Board 1.7.22


PIG-3585 Implement union in Tez

Review Request #15931 - Created Dec. 1, 2013 and submitted

Cheolsoo Park
PIG-3585
Reviewers
pig
abain, daijy, mwagner, rohini
pig-git
This patch implements union as follows: load vertices -> broadcast edges -> union vertex.

Th changes include:
* In the front-end, TezCompiler converts POUnion into a new vertex and connects it to its predecessors with broadcast edges.
* In the back-end, a new POPackage class called POBroadcastTezLoad is added. This classes implements TezLoad interface, and it pulls every record from ShuffledUnorderedKVInputs in order and unions them.
* New e2e test case is added.
* ant test-tez passes.
* All e2e tests pass.
Review request changed
Updated (Dec. 1, 2013, 11:14 p.m.)
  • changed from pending to submitted