Review Board 1.7.22

SQOOP-949: Allow Sqoop to build on Windows OS

Review Request #9961 - Created March 15, 2013 and updated

Ahmed El Baz
Allow Sqoop to build on Windows OS
Associated Jira:

Sqoop fails to build on Windows due to dependencies on Linux scripts like and which fail to build on Windows without Cygwin.

The patch does the following:
- provide Windows scripts src/scripts/create-tool-scripts.cmd and src/scripts/write-version-info.cmd which perform the same logic as the .sh scripts on Linux.
- Update build.xml to invoke the appropriate script based on the host OS.
- Skip real-docs target on Windows OS builds since as it invokes Linux specific "make" executable
Verified builds are successful on both Linux and Windows, and Unit tests are fully passing on Linux. Fixes to unit tests on Windows will go in a separate patch
Description From Last Updated Status
Is there a need to rename this? Jarek Cecho March 15, 2013, 7:50 p.m. Open
I've missed the place where the sqoop.cmd gets generated, would you mind pointing me? Jarek Cecho March 15, 2013, 7:50 p.m. Open
Posted (March 15, 2013, 7:15 p.m.)
Looks good to me.   Thanks for creating these batch scripts.   I will test this on my windows VM.   One quick question?   WHy don't you redirect the error output from the tool list generation to NUL (the equivalent of /dev/null) instead of a file.  Don't we have to clean it up at the end otherwise?
  1. Thank you Venkat for taking a look.
    The main purpose was debugging the scripts in case of errors. The file is overridden with every build.
  2. Thanks Ahmed.   It would be good to not have debug output statements or files around - but windows being new to many may be we can have it. 
  3. Thank you Venkat. In fact I agree with you especially that errors are not much expected in this scenario, so redirection can be enabled on demand in case of errors. Also it will be consistent with the behavior in Linux.
    I have updated the patch based on this feedback.
    Thank you
  4.  +1 looks good.
    Can you please also update the JIRA and add the Review board link as a web link than as a comment?
Posted (March 15, 2013, 7:50 p.m.)
Hi Ahmed,
thank you very much for all your work to get Sqoop compiled on Windows. I'm wondering whether there is need to document any extra steps or limitations?
build.xml (Diff revision 1)
Is there a need to rename this?
  1. Thank you Jarek for your comments.
    Yes in fact I have wrapped the doc.uptodate in a condition "skip-real-docs" which adds an OS constraint that host OS is not Windows. As I mentioned in the Jira, the real-docs target is Linux specific, hence the need to introduce the wrapper condition.
    I verified Linux builds are not affected
src/scripts/create-tool-scripts.cmd (Diff revision 1)
Nit: Trailing white space characters
  1. Fixed. Thank you :)
src/scripts/tool-script.cmd.template (Diff revision 1)
I've missed the place where the sqoop.cmd gets generated, would you mind pointing me?
  1. This change is the first part of getting Sqoop to work on Windows, which handles build problems, and provides build scripts.
    I will be posting another Jira for runtime scripts like sqoop.cmd and configure-sqoop.cmd, and also a fix for unit tests on Windows.
Posted (March 22, 2013, 6:15 a.m.)
Hi Ahmed

I applied this to my local repo and can build.   Thanks for the patch.   Couple of quick points.   May be the MSYY GIt issue but the patch did not apply cleanly (the new files were created on the project rootdir instead of under src/scripts directory.   May be a git format-patch would work better on Windows?   Not sure if others also hit similar issues

I think this patch is good to go, butI thought  we wanted to collect all of tge patches in lieu of the temporary branch approach.
Ship it!
Posted (April 6, 2013, 12:39 a.m.)
I think this patch is good.   I did not click ship-it before