Review Board 1.7.22


New logical plan fails when I have complex data types from udf

Review Request #526 - Created March 25, 2011 and submitted

Daniel Dai
PIG-1868
Reviewers
pig
thejas
pig
The new logical plan fails when I have complex data types returning from my eval function.

The below is my script :

register myudf.jar;   
B1 = load 'myinput' as (id:chararray,ts:int,url:chararray);
B2 = group B1 by id;
B = foreach B2 {
 Tuples = order B1 by ts;
 generate Tuples;
};
C1 = foreach B generate TransformToMyDataType(Tuples,-1,0,1) as seq: { t: ( previous, current, next ) };
C2 = foreach C1 generate FLATTEN(seq);
C3 = foreach C2 generate  current.id as id;
dump C3;

On C3 it fails with below message :

Couldn't find matching uid -1 for project (Name: Project Type: bytearray Uid: 45 Input: 0 Column: 1)

The below is the describe on C1 ;

C1: {seq: {t: (previous: (id: chararray,ts: int,url: chararray),current: (id: chararray,ts: int,url: chararray),next: (id: chararray,ts: int,url: chararray))}}

The script works if I turn off new logical plan or use Pig 0.7.
test-patch:
     [exec] +1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.

Unit test:
    all pass

End-to-end test:
    all pass
Review request changed
Updated (March 26, 2011, 10:56 p.m.)
  • 
    	  

    test-patch:
         [exec] +1 overall.  
         [exec] 
         [exec]     +1 @author.  The patch does not contain any @author tags.
         [exec] 
         [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
         [exec] 
         [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
         [exec] 
         [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
         [exec] 
         [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
         [exec] 
         [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
    
    Unit test:
        all pass
    
    End-to-end test:
        all pass