slave was crashed while using Row-based replication and merge engine

[yinfeng]

Percona Version:5.5.18

backtrace:

#0 0x000000000079400f in field_metadata (this=0x7f85f00087c0, thd=0x7f85f0000990, rli=0x157c37d8, table=0x7f85f0013010, conv_table_var=0x7f86172694c8)
    at /5.5.18/sql/rpl_utility.h:129
#1 table_def::compatible_with (this=0x7f85f00087c0, thd=0x7f85f0000990, rli=0x157c37d8, table=0x7f85f0013010, conv_table_var=0x7f86172694c8)
    at /5.5.18/sql/rpl_utility.cc:788
#2 0x0000000000725e51 in Rows_log_event::do_apply_event (this=0x7f85f000dd40, rli=0x157c37d8)
    at /5.5.18/sql/log_event.cc:7667
#3 0x0000000000513b85 in apply_event (ev=0x7f85f000dd40, thd=<value optimized out>, rli=0x157c37d8)
    at /5.5.18/sql/log_event.h:1135
#4 apply_event_and_update_pos (ev=0x7f85f000dd40, thd=<value optimized out>, rli=0x157c37d8) at /5.5.18/sql/slave.cc:2351
#5 0x0000000000518f4b in exec_relay_log_event (thd=0x7f85f0000990, rli=0x157c37d8) at /5.5.18/sql/slave.cc:2511
#6 0x000000000051a082 in handle_slave_sql (arg=0x157c2490) at /5.5.18/sql/slave.cc:3329
#7 0x000000377ce077e1 in start_thread () from /lib64/libpthread.so.0
#8 0x000000377cae68ed in clone () from /lib64/libc.so.6

this bug seems related to http://bugs.mysql.com/bug.php?id=47103. As mentioned there, the fix for bug #47103 have not been backported into 5.5 series

[Alexey Kopytov (akopytov)]

Bug #873044 was marked as a duplicate of this one.

[yinfeng (yinfeng-zwx)]

This bug was fixed in MySQL5.5.24, but not mentioned in changelog.

[Alexey Kopytov (akopytov)]

Thanks for the update. Indeed, the fix has been backported to 5.5 upstream with the following revision:

        revno: 3097.19.1
        committer: Luis Soares <email address hidden>
        branch nick: mysql-5.5-security
        timestamp: Fri 2012-02-24 16:07:43 +0000
        message:
          Bug#13693012: SLAVE CRASHING ON INSERT STATEMENT WITH MERGE TABLE

          PROBLEM: After WL 4144, when using MyISAM Merge tables, the routine
          open_and_lock_tables will append to the list of tables to lock, the
          base tables that make up the MERGE table. This has two side-effects in
          replication:

            1. On the master side, we log additional table maps for the base
               tables, since they appear in the list of locked tables, even
               though we don't really use them at the slave.

            2. On the slave side, when opening a MERGE table while applying a
               ROW event, additional tables are appended to the list of tables
               to lock.

          Side-effect #1 is not harmful. It's just that when using MyISAM Merge
          tables a few table maps more may be logged.

          Side-effect #2, is harmful, because the list rli->tables_to_lock is an
          extended structure from TABLE_LIST in which the extra fields are
          filled from the table maps that are processed. Since
          open_and_lock_tables appends tables to the list after all table map
          events have been processed we end up with entries without
          replication/table map data on them. Thus when trying to access that
          info for these extra tables, the server will crash.

          SOLUTION: We fix side-effect #2 by making sure that we access the
          replication part of the structure for those in the list that were
          accounted for when processing the correspondent table map events. All
          in all, we never go beyond rli->tables_to_lock_count.

          We also deploy an assertion when clearing rli->tables_to_lock, making
          sure that the base tables are not in the list anymore (were closed in
          close_thread_tables).

The revision was merged into 5.5.24.

[Mario Splivalo (mariosplivalo)]

This bug seems to be solved in current 5.5 release (5.5.31 at the time of this writing), but I'm hitting most likely related issue with cascading replication - second slave in chain crashes in the same manner as described in bug #873044. Full bug report is here: https://bugs.launchpad.net/percona-server/+bug/1191263

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-2772