Review Board 1.7.22


SQOOP-907 Print out queries executed by Sqoop on mapreduce job side

Review Request #9580 - Created Feb. 25, 2013 and submitted

Jarek Cecho
SQOOP-907
Reviewers
Sqoop
sqoop-trunk
We were already printing executed query on mapreduce side in debug mode, however as the mapreduce job do not inherit --verbose parameter, it was never printed. I've promote the message into info state. I've also added printing of current split.
Unit tests seems to be passing and I've tried it out on real cluster.
src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
Revision 4c03c0d7061a5202bdfb4827e261743fd228689c New Change
[20] 104 lines
[+20] [+] protected ResultSet executeQuery(String query) throws SQLException {
105
    if (fetchSize != null) {
105
    if (fetchSize != null) {
106
      LOG.debug("Using fetchSize for next query: " + fetchSize);
106
      LOG.debug("Using fetchSize for next query: " + fetchSize);
107
      statement.setFetchSize(fetchSize);
107
      statement.setFetchSize(fetchSize);
108
    }
108
    }
109

    
   
109

   
110
    LOG.debug("Executing query: " + query);
110
    LOG.info("Executing query: " + query);
111
    return statement.executeQuery();
111
    return statement.executeQuery();
112
  }
112
  }
113

    
   
113

   
114
  /** Returns the query for selecting the records,
114
  /** Returns the query for selecting the records,
115
   * subclasses can override this for custom behaviour.*/
115
   * subclasses can override this for custom behaviour.*/
[+20] [20] 115 lines
[+20] [+] public boolean nextKeyValue() throws IOException {
231
      if (value == null) {
231
      if (value == null) {
232
        value = createValue();
232
        value = createValue();
233
      }
233
      }
234
      if (null == this.results) {
234
      if (null == this.results) {
235
        // First time into this method, run the query.
235
        // First time into this method, run the query.

    
   
236
        LOG.info("Working on split: " + split);
236
        this.results = executeQuery(getSelectQuery());
237
        this.results = executeQuery(getSelectQuery());
237
      }
238
      }
238
      if (!results.next()) {
239
      if (!results.next()) {
239
        return false;
240
        return false;
240
      }
241
      }
[+20] [20] 79 lines
src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java
Revision 140517a7659d18a803a5fd06f60cb07acb201074 New Change
 
  1. src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java: Loading...
  2. src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBInputFormat.java: Loading...