Sporadic crash / valgrind warning in register_field_in_read_map() with semijoin

[Philip Stoev]

The attached test case produces the following warning. Unfortunately, no simplification is available for sporadically crashing valgrind bugs. If you can convert the warning into a repeatable crash or an assertion, I will be able to provide a simpler test case.

==3574== Thread 4:
==3574== Invalid read of size 4
==3574== at 0x81CB8EB: Item_field::register_field_in_read_map(unsigned char*) (item.cc:851)
==3574== by 0x81E2313: Item::walk(bool (Item::*)(unsigned char*), bool, unsigned char*) (item.h:971)
==3574== by 0x8219754: Item_equal::walk(bool (Item::*)(unsigned char*), bool, unsigned char*) (item_cmpfunc.cc:5721)
==3574== by 0x8245F75: Item_subselect::walk(bool (Item::*)(unsigned char*), bool, unsigned char*) (item_subselect.cc:526)
==3574== by 0x81E3E39: Item_ref::walk(bool (Item::*)(unsigned char*), bool, unsigned char*) (item.h:2629)
==3574== by 0x81F45AB: Item_func::walk(bool (Item::*)(unsigned char*), bool, unsigned char*) (item_func.cc:276)
==3574== by 0x8215F5B: Item_cond::walk(bool (Item::*)(unsigned char*), bool, unsigned char*) (item_cmpfunc.cc:4252)
==3574== by 0x83EA3EB: find_all_keys(st_sort_param*, SQL_SELECT*, unsigned char**, st_io_cache*, st_io_cache*, st_io_cache*) (filesort.cc:551)
==3574== by 0x83E94EE: filesort(THD*, st_table*, st_sort_field*, unsigned int, SQL_SELECT*, unsigned long long, bool, unsigned long long*) (filesort.cc:240)
==3574== by 0x83447D8: create_sort_index(THD*, JOIN*, st_order*, unsigned long long, unsigned long long, bool) (sql_select.cc:18103)
==3574== by 0x83213B5: JOIN::exec() (sql_select.cc:2622)
==3574== by 0x8321E95: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:2900)
==3574== by 0x8319C66: handle_select(THD*, st_lex*, select_result*, unsigned long) (sql_select.cc:283)
==3574== by 0x82B4433: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:5110)
==3574== by 0x82AB1F0: mysql_execute_command(THD*) (sql_parse.cc:2250)
==3574== by 0x82B6A62: mysql_parse(THD*, char*, unsigned int, char const**) (sql_parse.cc:6110)
==3574== Address 0x5ab02a4 is 52 bytes inside a block of size 996 free'd
==3574== at 0x4005AB8: free (vg_replace_malloc.c:366)
==3574== by 0x877F07F: _myfree (safemalloc.c:335)
==3574== by 0x877E418: free_root (my_alloc.c:370)
==3574== by 0x833C5F8: free_tmp_table(THD*, st_table*) (sql_select.cc:14594)
==3574== by 0x824EE17: subselect_hash_sj_engine::cleanup() (item_subselect.cc:4357)
==3574== by 0x82452B5: Item_subselect::cleanup() (item_subselect.cc:130)
==3574== by 0x8245492: Item_in_subselect::cleanup() (item_subselect.cc:165)
==3574== by 0x8331A17: st_join_table::cleanup() (sql_select.cc:9611)
==3574== by 0x8332348: JOIN::cleanup(bool) (sql_select.cc:9892)
==3574== by 0x8332074: JOIN::join_free() (sql_select.cc:9812)
==3574== by 0x833CCF0: do_select(JOIN*, List<Item>*, st_table*, Procedure*) (sql_select.cc:14778)
==3574== by 0x8321668: JOIN::exec() (sql_select.cc:2679)
==3574== by 0x824C4FA: subselect_single_select_engine::exec() (item_subselect.cc:2968)
==3574== by 0x8246246: Item_subselect::exec() (item_subselect.cc:572)
==3574== by 0x82475A3: Item_singlerow_subselect::val_str(String*) (item_subselect.cc:1090)
==3574== by 0x831F7B0: JOIN::exec() (sql_select.cc:2147)

Optimizer switch for the particular example semijoin=ON,materialization=ON,firstmatch=OFF,loosescan=OFF

 bzr version-info
revision-id: <email address hidden>
date: 2011-09-26 23:54:00 +0300
build-date: 2011-09-27 16:41:56 +0300
revno: 3196
branch-nick: maria-5.3

[Sergey Petrunia (sergefp)]

A simplified testcase:

SET SESSION optimizer_switch='semijoin=ON,materialization=ON,firstmatch=OFF,loosescan=OFF';

SELECT
   alias1.col_int_nokey AS field1 ,
   (SELECT
      MAX( SQ2_alias1.col_varchar_key) AS SQ2_field1
    FROM
      t2 AS SQ2_alias1
    WHERE
       (SQ2_alias1.pk , SQ2_alias1.pk) IN
       (
          SELECT
             C_SQ1_alias2.col_int_nokey AS C_SQ1_field1,
             C_SQ1_alias1.col_int_key AS C_SQ1_field2
          FROM
          (
             t3 AS C_SQ1_alias1,
             t3 AS C_SQ1_alias2,
             t1 AS C_SQ1_alias3
          )
          WHERE C_SQ1_alias3.col_int_key > 98 AND
                C_SQ1_alias3.pk = C_SQ1_alias2.pk AND
                C_SQ1_alias3.col_varchar_key = C_SQ1_alias2.col_varchar_key
          GROUP BY C_SQ1_field1 , C_SQ1_field2
       )
   ) AS field3
FROM
  t3 AS alias1,
  t4 AS alias2,
  t1 AS alias3

GROUP BY
  field1, field3
HAVING
  field3 < 'e' AND field1 < 242
ORDER BY
  alias1.col_datetime_key ASC , field3 ASC, field1, field3
LIMIT 100 OFFSET 9 ;

drop table t1,t2,t3,t4;

[Sergey Petrunia (sergefp)]

EXPLAIN:
+----+-------------+--------------+--------+-------------------------------------+--------------+---------+--------------------+------+-------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+--------+-------------------------------------+--------------+---------+--------------------+------+-------------------------------------------------+
| 1 | PRIMARY | alias1 | ALL | NULL | NULL | NULL | NULL | 20 | Using temporary; Using filesort |
| 1 | PRIMARY | alias3 | index | NULL | PRIMARY | 4 | NULL | 20 | Using index; Using join buffer (flat, BNL join) |
| 1 | PRIMARY | alias2 | index | NULL | PRIMARY | 4 | NULL | 100 | Using index; Using join buffer (flat, BNL join) |
| 2 | SUBQUERY | SQ2_alias1 | system | PRIMARY | NULL | NULL | NULL | 1 | |
| 2 | SUBQUERY | <subquery3> | eq_ref | distinct_key | distinct_key | 8 | const,const | 1 | |
| 3 | SUBQUERY | C_SQ1_alias3 | range | PRIMARY,col_int_key,col_varchar_key | col_int_key | 4 | NULL | 3 | Using where; Using temporary |
| 3 | SUBQUERY | C_SQ1_alias1 | index | NULL | col_int_key | 4 | NULL | 20 | Using index; Using join buffer (flat, BNL join) |
| 3 | SUBQUERY | C_SQ1_alias2 | eq_ref | PRIMARY,col_varchar_key | PRIMARY | 4 | j0.C_SQ1_alias3.pk | 1 | Using where |
+----+-------------+--------------+--------+-------------------------------------+--------------+---------+--------------------+------+-------------------------------------------------+
8 rows in set (0.02 sec)

[Sergey Petrunia (sergefp)]

An outline of the crashing scenario:

== Query ==
The query has this structure:

select
  single_row_subselect (select with non-mergeable
                         semi-join subselect) as field3
from
  ...
group by
  ... field3
having
  field3 < 'e' and ...
order by
  ..., field3, ...
limit ..

Things to note:
- single_row_subselect is uncorrelated
- it is used in select list, group list, order by list.

== Execution ==
Execution proceeds as follows:
- single_row_subselect is put into top-level select's
  exec_const_order_group_cond list.
- JOIN::exec calls cur_const_item->val_str, which evaluates the
  single_row_subselect:

  = single_row_subselect has a non-mergeable semi-join which is executed
    as materialized join tab (a temporary table is created, and the
    IN-equality is injected into subquery's WHERE condition)

  = After the subquery has finished executing, the code for materialized
    join_tab makes this call:

        table->pos_in_table_list->jtbm_subselect->cleanup();

    the stack trace at this point looks as follows:

        #1 in Item_in_subselect::cleanup
        #2 in st_join_table::cleanup
        #3 in JOIN::cleanup
        #4 in JOIN::join_free
        #5 in do_select
        #6 in JOIN::exec
        #7 in subselect_single_select_engine::exec
        #8 in Item_subselect::exec
        #9 in Item_singlerow_subselect::val_str
        ...

  = single_row_subselect execution is completed. The code for top-level select
    makes multiple attempts to evaluate the single_row_subselect but since it
    is uncorrelated, any subsequent attempts only fetch the cached value.

- evenually, execution of upper select reaches filesort() which calls
  find_all_keys() which runs this line:

    select->cond->walk(&Item::register_field_in_read_map, 1,
                       (uchar*) sort_form);

  where
     select->cond = ((`field3` < 'e') and (`field1` < 242))

  where field3 is the single_row_subselect.

  field3->Item_subselect::walk is called which will at some point invoke

     Item_field::register_field_in_read_map

  for the field object that refers to already-freed temporary table.