Review Board 1.7.22


OOZIE-667 Change the way Oozie brings in Hadoop JARs into the build

Review Request #3726 - Created Feb. 1, 2012 and submitted

Alejandro Abdelnur
trunk
OOZIE-667
Reviewers
oozie
oozie
The current mechanism allow to easily package/use new versions of hadoop without complicating Oozie's POMs.

mr1 & mr2 profiles are gone from oozie, now it refers to one of the versions in hadooplibs. 

New maven modules under hadooplib define the hadoop-client/hadoop-test POMs for different hadoop versions.

Note that because of HADOOP-8009, hadoop will start providing a hadoop-client artifact (even for already released versions), still we'll need the corresponding hadooplibs module to be able to use the assembly (as it is done as part of this patch) to pull into the oozie distro the JARs for the supported/tested versions of Hadoop required for the client side.

Note that this can be used tosimplify the logic of addtowar.sh that won't have to be aware of the JARs deps of different versions of Hadoop or of hadoop JARs at all.

Tested with 1.0.0 and 0.24.0-SNAPSHOTs (several testcases failures here but that is still work in progress for the mr2 integration)
Review request changed
Updated (Feb. 3, 2012, 5:44 a.m.)
changes as per Mohammad's comments
Ship it!
Posted (Feb. 3, 2012, 8:38 a.m.)
+1 after addressing the doc comments for clarifications.
It is not necessary to use the oozie-bundled hadoop libs. For example, H 0.24.2 is released and we are late or not packaging for some reason. There should be a way, for user to collect the necessary hadoop jars and put it into libext/.
I know this is obvious. Just to mention in some way that user can bring there own hadoop library and copy it into libext.