Wrong result with index_condition_pushdown + GROUP BY/ORDER BY

[Philip Stoev]

This query:

SELECT a, MIN(d)
FROM t1
WHERE c = 'x'
AND d > 'x'
GROUP BY a;

when executed with ICP, returns rows that do not match the second, nonindexed part, of the WHERE predicate:

+------+--------+
| a | MIN(d) |
+------+--------+
| 2 | x |
| 5 | x |
+------+--------+

explain:

+----+-------------+-------+------+---------------+------+---------+-------+------+---------------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+-------+------+---------------------------------------------------------------------+
| 1 | SIMPLE | t1 | ref | c | c | 4 | const | 1 | Using index condition; Using where; Using temporary; Using filesort |
+----+-------------+-------+------+---------------+------+---------+-------+------+---------------------------------------------------------------------+

 bzr version-info
revision-id: <email address hidden>
date: 2011-10-06 01:21:15 +0400
build-date: 2011-10-07 17:34:34 +0300
revno: 3213

minimal switch: index_condition_pushdown=ON
full switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,index_condition_pushdown=off,derived_merge=off,derived_with_keys=off,firstmatch=off,loosescan=off,materialization=off,in_to_exists=on,semijoin=off,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=off,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=off,semijoin_with_cache=off,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on
join_cache_level = 1

test case:

DROP TABLE t1;
CREATE TABLE t1 ( a int, c varchar(1), d varchar(1), KEY c (c));
INSERT INTO t1 VALUES (2,'x','x'),(5,'x','x');

SET SESSION optimizer_switch='index_condition_pushdown=on';
SELECT a, MIN(d)
FROM t1
WHERE c = 'x'
AND d > 'x'
GROUP BY a;

[Philip Stoev (pstoev-askmonty)]

Here is another example:

DROP TABLE IF EXISTS lineitem;
CREATE TABLE lineitem ( l_orderkey int(11) NOT NULL , l_linenumber int(11) NOT NULL , PRIMARY KEY (l_orderkey,l_linenumber)) ;
INSERT IGNORE INTO lineitem VALUES (514,4),(551,3),(576,1),(576,2),(576,3),(576,4),(577,1),(577,2),(578,1),(578,2),(579,1),(579,2),(579,3),(579,4),(579,5),(579,6),(580,1),(580,2),(580,3),(581,1),(581,2),(581,3),(581,4),(582,1),(582,2),(582,3),(582,4);

DROP TABLE IF EXISTS orders;
CREATE TABLE orders ( o_orderkey int(11) NOT NULL , o_custkey int(11), KEY (o_custkey), PRIMARY KEY (o_orderkey)) ;
INSERT IGNORE INTO orders VALUES (513,7),(514,8);

DROP TABLE IF EXISTS customer;
CREATE TABLE customer ( c_custkey int(11) NOT NULL , c_nationkey int(11), KEY (c_nationkey), PRIMARY KEY (c_custkey)) ;
INSERT IGNORE INTO customer VALUES (8,17),(10,5);

DROP TABLE IF EXISTS supplier;
CREATE TABLE supplier ( s_nationkey int(11), KEY (s_nationkey)) ;
INSERT IGNORE INTO supplier VALUES (17);

SET SESSION optimizer_switch='index_condition_pushdown=ON';
EXPLAIN SELECT c_custkey FROM lineitem JOIN orders ON ( l_orderkey = o_orderkey ) JOIN customer ON ( o_custkey = c_custkey ) JOIN nation ON ( c_nationkey = n_nationkey ) JOIN supplier ON ( s_nationkey = n_nationkey ) WHERE c_custkey IN ( 170 , 137 ) group by 1;

Note that the EXPLAIN of the query does not include any reference to ICP, and yet the query returns a wrong result. Every use of any optimization, ICP included, must be reflected in EXPLAIN. Otherwise, it is very difficult to test.

[Philip Stoev (pstoev-askmonty)]

And here is one with ORDER BY:

CREATE TABLE lineitem ( l_orderkey int NOT NULL , l_linenumber int NOT NULL , PRIMARY KEY (l_orderkey,l_linenumber)) ;
INSERT INTO lineitem VALUES (514,4),(582,3),(582,4);

CREATE TABLE orders ( o_orderkey int NOT NULL , o_custkey int, KEY (o_custkey), PRIMARY KEY (o_orderkey)) ;
INSERT INTO orders VALUES (484,7),(485,11),(486,7),(487,11),(512,7),(513,7),(514,8);

CREATE TABLE customer ( c_custkey int NOT NULL , c_nationkey int, c_acctbal double, KEY (c_nationkey), PRIMARY KEY (c_custkey)) ;
INSERT INTO customer VALUES (8,17,'6819.74'),(10,5,'2753.54');

CREATE TABLE nation ( n_nationkey int NOT NULL , PRIMARY KEY (n_nationkey)) ;
INSERT INTO nation VALUES (17),(19);

CREATE TABLE supplier ( s_nationkey int, KEY (s_nationkey)) ;
INSERT INTO supplier VALUES (17);

SET SESSION optimizer_switch='index_condition_pushdown=ON';
SELECT c_acctbal FROM lineitem JOIN orders ON ( l_orderkey = o_orderkey ) JOIN customer ON ( o_custkey = c_custkey ) JOIN nation ON ( c_nationkey = n_nationkey ) JOIN supplier ON ( s_nationkey = n_nationkey ) WHERE c_acctbal < 200 ORDER BY c_acctbal;