Crash in JOIN_CACHE::get_offset with join_cache_level=6 in maria 5.3

[Philip Stoev]

The following query:

SELECT MAX( table2 . `col_varchar_key` ) AS field1 FROM ( C AS table1 STRAIGHT_JOIN ( ( B AS table2 RIGHT JOIN CC AS table3 ON (( table3 .`pk` > table2 . `pk` ) AND (table3 .`pk` <= table2 . `col_int_key` ) ) ) ) ON (( table3 . `col_varchar_key` = table2 . `col_varchar_nokey` ) AND ( table3 . `pk` = table2 . `col_int_key` ) ) ) HAVING field1 <= 'm';

 causes the following crash:

# 2010-08-24T10:45:49 #3 0x0828048a in handle_segfault (sig=11) at mysqld.cc:2706
# 2010-08-24T10:45:49 #4 <signal handler called>
# 2010-08-24T10:45:49 #5 0x082dbc46 in JOIN_CACHE::get_offset (this=0x99e1e310, ofs_sz=1, ptr=0xeea394f <Address 0xeea394f out of bounds>) at sql_select.h:455
# 2010-08-24T10:45:49 #6 0x082dbd22 in JOIN_CACHE::get_rec_length (this=0x99e1e310, ptr=0xeea394f <Address 0xeea394f out of bounds>) at sql_select.h:634
# 2010-08-24T10:45:49 #7 0x082d90ed in JOIN_CACHE::read_referenced_field (this=0x99e1e310, copy=0x99e1e3fc, rec_ptr=0xeea3950 <Address 0xeea3950 out of bounds>, len=0x9a1dca00)
# 2010-08-24T10:45:49 at sql_join_cache.cc:1511
# 2010-08-24T10:45:49 #8 0x082da6c3 in JOIN_CACHE_BKA::get_next_key (this=0x99e1e468, key=0x9a1dcaa8) at sql_join_cache.cc:2512
# 2010-08-24T10:45:49 #9 0x082d9f1d in bka_range_seq_next (rseq=0x99e1e468, range=0x9a1dcaa8) at sql_join_cache.cc:2180
# 2010-08-24T10:45:49 #10 0x08499998 in DsMrr_impl::dsmrr_fill_key_buffer (this=0x99e58ee4) at multi_range_read.cc:859
# 2010-08-24T10:45:49 #11 0x084989a7 in DsMrr_impl::dsmrr_init (this=0x99e58ee4, h_arg=0x99e58b38, seq_funcs=0x9a1dcbfc, seq_init_param=0x99e1e468, n_ranges=20, mode=129,
# 2010-08-24T10:45:49 buf=0x99e1e4ec) at multi_range_read.cc:458
# 2010-08-24T10:45:49 #12 0x0866a751 in ha_innobase::multi_range_read_init (this=0x99e58b38, seq=0x9a1dcbfc, seq_init_param=0x99e1e468, n_ranges=20, mode=129, buf=0x99e1e4ec)
# 2010-08-24T10:45:49 at handler/ha_innodb.cc:11213
# 2010-08-24T10:45:49 #13 0x082da42c in JOIN_CACHE_BKA::init_join_matching_records (this=0x99e1e468, seq_funcs=0x9a1dcbfc, ranges=20) at sql_join_cache.cc:2396
# 2010-08-24T10:45:49 #14 0x082da17d in JOIN_CACHE_BKA::join_matching_records (this=0x99e1e468, skip_last=false) at sql_join_cache.cc:2303
# 2010-08-24T10:45:49 #15 0x082d9352 in JOIN_CACHE::join_records (this=0x99e1e468, skip_last=false) at sql_join_cache.cc:1639
# 2010-08-24T10:45:49 #16 0x082d9484 in JOIN_CACHE::join_records (this=0x99e1e310, skip_last=false) at sql_join_cache.cc:1685
# 2010-08-24T10:45:49 #17 0x0831f382 in sub_select_cache (join=0x99e25260, join_tab=0x99e1dd50, end_of_records=true) at sql_select.cc:12878
# 2010-08-24T10:45:49 #18 0x0831f59b in sub_select (join=0x99e25260, join_tab=0x99e1db90, end_of_records=true) at sql_select.cc:13040
# 2010-08-24T10:45:49 #19 0x0831eb55 in do_select (join=0x99e25260, fields=0x99e2a02c, table=0x0, procedure=0x0) at sql_select.cc:12636
# 2010-08-24T10:45:49 #20 0x08304ed7 in JOIN::exec (this=0x99e25260) at sql_select.cc:2355
# 2010-08-24T10:45:49 #21 0x0830560d in mysql_select (thd=0xbcc5828, rref_pointer_array=0xbcc72c0, tables=0x99e9be08, wild_num=0, fields=..., conds=0x0, og_num=0, order=0x0,
# 2010-08-24T10:45:49 group=0x0, having=0x99e21938, proc_param=0x0, select_options=2147764736, result=0x99ec7b60, unit=0xbcc6f04, select_lex=0xbcc71a4) at sql_select.cc:2556
# 2010-08-24T10:45:49 #22 0x082fdbff in handle_select (thd=0xbcc5828, lex=0xbcc6ea8, result=0x99ec7b60, setup_tables_done_option=0) at sql_select.cc:276
# 2010-08-24T10:45:49 #23 0x0829c1a4 in execute_sqlcom_select (thd=0xbcc5828, all_tables=0x99e9be08) at sql_parse.cc:5081
# 2010-08-24T10:45:49 #24 0x08292b7c in mysql_execute_command (thd=0xbcc5828) at sql_parse.cc:2265
# 2010-08-24T10:45:49 #25 0x0829e365 in mysql_parse (thd=0xbcc5828,
# 2010-08-24T10:45:49 inBuf=0x99e2d6c8 "SELECT MAX( table2 . `col_varchar_key` ) AS field1 FROM ( C AS table1 STRAIGHT_JOIN ( ( B AS table2 RIGHT JOIN CC AS table3 ON (( table3 .`pk` > table2 . `pk` ) AND (table3 .`pk` <= table2 . `col_int_key` ) ) ) ) ON (( table3 . `col_varchar_key` = table2 . `col_varchar_nokey` ) AND ( table3 . `pk` = table2 . `col_int_key` ) ) ) HAVING field1 <= 'm'", length=357, found_semicolon=0x9a1de230) at sql_parse.cc:6027
# 2010-08-24T10:45:49 #26 0x08290632 in dispatch_command (command=COM_QUERY, thd=0xbcc5828, packet=0xbcc7849 "", packet_length=359) at sql_parse.cc:1184
# 2010-08-24T10:45:49 #27 0x0828fb20 in do_command (thd=0xbcc5828) at sql_parse.cc:890
# 2010-08-24T10:45:49 #28 0x0828cb58 in handle_one_connection (arg=0xbcc5828) at sql_connect.cc:1153
# 2010-08-24T10:45:49 #29 0x00a08919 in start_thread () from /lib/libpthread.so.0
# 2010-08-24T10:45:49 #30 0x001ede5e in clone () from /lib/libc.so.6

bzr version-info:

revision-id: <email address hidden>
date: 2010-08-19 19:52:58 +0200
build-date: 2010-08-24 10:48:53 +0300
revno: 2821
branch-nick: maria-5.3-dsmrr-cpk

[Philip Stoev (pstoev-askmonty)]

Test case:

--source include/have_innodb.inc

SET SESSION join_cache_level=6;

CREATE TABLE `CC` (
  `pk` int(11) NOT NULL AUTO_INCREMENT,
  `col_int_key` int(11) DEFAULT NULL,
  `col_varchar_key` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`pk`),
  KEY `col_varchar_key` (`col_varchar_key`,`col_int_key`)
) ENGINE=InnoDB AUTO_INCREMENT=30 DEFAULT CHARSET=latin1;
INSERT INTO `CC` VALUES (29,4,'c');
INSERT INTO `CC` VALUES (17,0,'z');
CREATE TABLE `C` (
  `pk` int(11) NOT NULL AUTO_INCREMENT,
  `col_int_key` int(11) DEFAULT NULL,
  `col_varchar_key` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`pk`),
  KEY `col_varchar_key` (`col_varchar_key`,`col_int_key`)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=latin1;
INSERT INTO `C` VALUES (11,0,NULL);
INSERT INTO `C` VALUES (18,5,'y');
CREATE TABLE `B` (
  `pk` int(11) NOT NULL AUTO_INCREMENT,
  `col_int_key` int(11) DEFAULT NULL,
  `col_varchar_key` varchar(1) DEFAULT NULL,
  PRIMARY KEY (`pk`),
  KEY `col_varchar_key` (`col_varchar_key`,`col_int_key`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
INSERT INTO `B` VALUES (1,7,'f');

SELECT table2 .`col_varchar_key`
FROM C STRAIGHT_JOIN ( B table2 JOIN CC table3 ON table2 .`pk` ) ON table3 .`col_varchar_key` AND table3 .`pk` = table2 .`col_int_key` ;

[Philip Stoev (pstoev-askmonty)]

Valgrind:

==12613== Thread 23:
==12613== Invalid read of size 1
==12613== at 0x8458D1E: JOIN_CACHE::get_offset(unsigned int, unsigned char*) (sql_select.h:455)
==12613== by 0x8458F85: JOIN_CACHE::get_rec_length(unsigned char*) (sql_select.h:634)
==12613== by 0x8453032: JOIN_CACHE::read_referenced_field(st_cache_field*, unsigned char*, unsigned int*) (sql_join_cache.cc:1511)
==12613== by 0x8455FD2: JOIN_CACHE_BKA::get_next_key(unsigned char**) (sql_join_cache.cc:2512)
==12613== by 0x8455077: bka_range_seq_next(void*, st_key_multi_range*) (sql_join_cache.cc:2180)
==12613== by 0x87F9BC7: DsMrr_impl::dsmrr_fill_key_buffer() (multi_range_read.cc:859)
==12613== by 0x87F78E8: DsMrr_impl::dsmrr_init(handler*, st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (multi_range_read.cc:458)
==12613== by 0x8B45354: ha_innobase::multi_range_read_init(st_range_seq_if*, void*, unsigned int, unsigned int, st_handler_buffer*) (ha_innodb.cc:11213)
==12613== by 0x8455ADD: JOIN_CACHE_BKA::init_join_matching_records(st_range_seq_if*, unsigned int) (sql_join_cache.cc:2396)
==12613== by 0x8455528: JOIN_CACHE_BKA::join_matching_records(bool) (sql_join_cache.cc:2303)
==12613== by 0x845359F: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:1639)
==12613== by 0x84E1D47: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:12898)
==12613== by 0x845466B: JOIN_CACHE::generate_full_extensions(unsigned char*) (sql_join_cache.cc:1944)
==12613== by 0x84541A9: JOIN_CACHE_BNL::join_matching_records(bool) (sql_join_cache.cc:1836)
==12613== by 0x845359F: JOIN_CACHE::join_records(bool) (sql_join_cache.cc:1639)
==12613== by 0x84E1A6F: sub_select_cache(JOIN*, st_join_table*, bool) (sql_select.cc:12878)

[Sergey Petrunia (sergefp)]

EXPLAIN of the offending query:

+----+-------------+--------+--------+---------------+-----------------+---------+-------------------------+------+--------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+--------+---------------+-----------------+---------+-------------------------+------+--------------------------------+
| 1 | SIMPLE | table2 | index | NULL | col_varchar_key | 9 | NULL | 1 | Using where; Using index |
| 1 | SIMPLE | C | index | NULL | PRIMARY | 4 | NULL | 2 | Using index; Using join buffer |
| 1 | SIMPLE | table3 | eq_ref | PRIMARY | PRIMARY | 4 | dsm1.table2.col_int_key | 1 | Using where; Using join buffer |
+----+-------------+--------+--------+---------------+-----------------+---------+-------------------------+------+--------------------------------+

[Sergey Petrunia (sergefp)]

The problem can be observed in mainline 5.3 code, as well. One needs to modify the query so that table3 is accessed via non-primary key:

MariaDB [dsm1]> alter table CC add key(pk);

and force the table2,C,table3 join order in the query:

MariaDB [dsm1]> explain SELECT table2 .`col_varchar_key` FROM B table2 STRAIGHT_JOIN C STRAIGHT_JOIN CC table3 force index(pk) WHERE table2 .`pk` AND table3 .`col_varchar_key` AND table3 .`pk` = table2 .`col_int_key`;
+----+-------------+--------+-------+---------------+-----------------+---------+-------------------------+------+--------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------+-------+---------------+-----------------+---------+-------------------------+------+--------------------------------+
| 1 | SIMPLE | table2 | index | NULL | col_varchar_key | 9 | NULL | 1 | Using where; Using index |
| 1 | SIMPLE | C | index | NULL | PRIMARY | 4 | NULL | 2 | Using index; Using join buffer |
| 1 | SIMPLE | table3 | ref | pk | pk | 4 | dsm1.table2.col_int_key | 1 | Using where; Using join buffer |
+----+-------------+--------+-------+---------------+-----------------+---------+-------------------------+------+--------------------------------+

Then, put a breakpoint at bka_range_seq_next():

  Breakpoint 1, bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2173

And run the SELECT:

MariaDB [dsm1]> SELECT table2 .`col_varchar_key` FROM B table2 STRAIGHT_JOIN C STRAIGHT_JOIN CC table3 force index(pk) WHERE table2 .`pk` AND table3 .`col_varchar_key` AND table3 .`pk` = table2 .`col_int_key`;

When the breakpoint is hit, we can observe that join buffer has two records:

  JOIN_CACHE_BKA::get_next_key (this=0xb1246f0, key=0xb124858) at sql_join_cache.cc:2464
(gdb) p this->records
  $1 = 2

Continue execution two times, so that we're stopped in the 3rd invocation of bka_range_seq_next().

(gdb) c
  Continuing.

  Breakpoint 1, bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2173
(gdb) c
  Continuing.

  Breakpoint 1, bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2173

At this point, one would expect that bka_range_seq_next() returns EOF, since all records in the buffer have already been enumerated.

(gdb) next
(gdb) next
(gdb) next
(gdb) next
(gdb) step
  JOIN_CACHE_BKA::get_next_key (this=0xb1246f0, key=0xb124858) at sql_join_cache.cc:2464
(gdb) p last_rec_pos - pos
  $6 = 0
(gdb) up
  #1 0x082f8257 in bka_range_seq_next (rseq=0xb1246f0, range=0xb124858) at sql_join_cache.cc:2177
(gdb) up
  #2 0x084d7936 in handler::multi_range_read_next (this=0xb1247b8, range_info=0x9e8765ec) at multi_range_read.cc:259
(gdb) up
  #3 0x084d8476 in DsMrr_impl::dsmrr_fill_buffer (this=0xb0ea340) at multi_range_read.cc:491
(gdb) p rowids_buf
  $7 = (uchar *) 0xb1459d2 '¥' <repeats 200 times>...

Remember this pointer ^^^^^^

(gdb) down
  #2 0x084d7936 in ha...

[Sergey Petrunia (sergefp)]

Igor, could you please take a look from BKA side?

[Philip Stoev (pstoev-askmonty)]

Ok point taken, I will re-verify bugs using the main tree + valgrind to avoid situations where crashes only happen on the feature tree due to some particular arrangement of memory accesses.