Review Board 1.7.22


PIG-2672 Optimize the use of DistributedCache

Review Request #14274 - Created Sept. 21, 2013 and updated

Aniket Mokashi
PIG-2672
Reviewers
pig
cheolsoo, Daniel, dvryaboy, julien, rohini
pig
added jar.cache.location option

 
trunk/conf/pig.properties
Revision 1564020 New Change
[20] 244 lines
[+20]
245
# pig.auto.local.input.maxbytes=100000000
245
# pig.auto.local.input.maxbytes=100000000
246

    
   
246

   
247
# When enabled, jobs won't create empty part files if no output is written. In this case
247
# When enabled, jobs won't create empty part files if no output is written. In this case
248
# PigOutputFormat will be wrapped with org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat.
248
# PigOutputFormat will be wrapped with org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat.
249
# pig.output.lazy=true
249
# pig.output.lazy=true

    
   
250

   

    
   
251
# Set this option to turn on additional jar caching for the user

    
   
252
# pig.user.cache.enabled=true

    
   
253

   

    
   
254
# This option defines location where additional jars are cached for the user.

    
   
255
# Additional jar will be cached under PIG_USER_CACHE_LOCATION/${user.name}/.pigcache

    
   
256
# and will be re-used across the jobs run by the user if the jar has not changed.

    
   
257
# pig.user.cache.location=/tmp
trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java
Revision 1564020 New Change
 
trunk/src/org/apache/pig/PigConfiguration.java
Revision 1564020 New Change
 
trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java
Revision 1564020 New Change
 
trunk/src/org/apache/pig/impl/io/FileLocalizer.java
Revision 1564020 New Change
 
trunk/test/org/apache/pig/test/TestMultiQueryLocal.java
Revision 1564020 New Change
 
trunk/test/org/apache/pig/test/TestPigServer.java
Revision 1564020 New Change
 
  1. trunk/conf/pig.properties: Loading...
  2. trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java: Loading...
  3. trunk/src/org/apache/pig/PigConfiguration.java: Loading...
  4. trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java: Loading...
  5. trunk/src/org/apache/pig/impl/io/FileLocalizer.java: Loading...
  6. trunk/test/org/apache/pig/test/TestMultiQueryLocal.java: Loading...
  7. trunk/test/org/apache/pig/test/TestPigServer.java: Loading...