Trafodion

Bug #1391271
Comment #0

Comment 0 for bug 1391271

Revision history for this message

Weishiun Tsai (wei-shiun-tsai) wrote on 2014-11-10:

During the QA table population with the v1107_0830 build on a Cloudera cluster, update statistics encountered this error 3 times. This OutOfOrderScannerNextException has not shown up for quite some time. Some recent changes in update statics may have caused the error to return again. This bug report is to document this problem, so that some research can be done in this area to figure out what in update statistics may have stressed hbase to return this error again.

SQL>update statistics for table trafodion.g_tpcds1x.inventory on every column sample;
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.G_TPCDS1X.INVENTORY encountered an error (8448) from statement Process_Query. [2014-11-08 13:01:55]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-705). Cause:
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
java.util.concurrent.FutureTask.report(FutureTask.java:122)
java.util.concurrent.FutureTask.get(FutureTask.java:188)
org.trafodion.sql.HBaseAccess.HTableClient.fetchRows(HTableClient.java:415)
. [2014-11-08 13:01:55]

SQL>update statistics for table trafodion.g_tpcds1x.store_sales on every column sample;
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.G_TPCDS1X.STORE_SALES encountered an error (8448) from statement Process_Query. [2014-11-08 13:14:29]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-705). Cause:
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
java.util.concurrent.FutureTask.report(FutureTask.java:122)
java.util.concurrent.FutureTask.get(FutureTask.java:188)
org.trafodion.sql.HBaseAccess.HTableClient.fetchRows(HTableClient.java:415)
. [2014-11-08 13:14:29]

SQL>update statistics for table cbase on every column sample random 5 percent;
*** ERROR[9200] UPDATE STATISTICS for table TRAFODION.G_WISC32.CBASE encountered an error (8448) from statement Process_Query. [2014-11-08 16:15:06]
*** ERROR[8448] Unable to access Hbase interface. Call to ExpHbaseInterface::nextRow returned error HBASE_ACCESS_ERROR(-705). Cause:
java.util.concurrent.ExecutionException: org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of OutOfOrderScannerNextException: was there a rpc timeout?
java.util.concurrent.FutureTask.report(FutureTask.java:122)
java.util.concurrent.FutureTask.get(FutureTask.java:188)
org.trafodion.sql.HBaseAccess.HTableClient.fetchRows(HTableClient.java:415)
. [2014-11-08 16:15:06]

During the QA table population with the v1107_0830 build on a Cloudera cluster, update statistics encountered this error 3 times.  This OutOfOrderScannerNextException has not shown up for quite some time.  Some recent changes in update statics may have caused the error to return again.  This bug report is to document this problem, so that some research can be done in this area to figure out what in update statistics may have stressed hbase to return this error again.