Regression test for lp:1019473 will cause abort due to BF-BF lock conflict in lock0lock.c::lock_rec_has_to_wait():
if (wsrep_trx_order_before(trx->mysql_thd,
lock2->trx->mysql_thd) &&
(type_mode & LOCK_MODE_MASK) == LOCK_X &&
(lock2->type_mode & LOCK_MODE_MASK) == LOCK_X)
{
/* exclusive lock conflicts are not accepted */
fprintf(stderr, "BF-BF X lock conflict\n");
lock_rec_print(stderr, lock2);
abort();
} else {
...
131226 3:27:03 [Note] WSREP: replay trx: DELETE FROM lp1019473 WHERE fid=4 AND
uid=4 86889
BF-BF lock conflict
RECORD LOCKS space id 0 page no 663 n bits 1056 index `uid` of table `test`.`lp1
019473` trx id 20C06 lock_mode X locks rec but not gap
Record lock, heap no 98 PHYSICAL RECORD: n_fields 2; compact format; info bits 3
2
0: len 4; hex 00000004; asc ;;
1: len 6; hex 0000000084ab; asc ;;
131226 3:27:03 [Note] WSREP: BF conflict, order: 86889 86892
BF-BF X lock conflict
RECORD LOCKS space id 0 page no 663 n bits 1056 index `uid` of table `test`.`lp1019473` trx id 20C06 lock_mode X locks rec but not gap
Record lock, heap no 98 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
0: len 4; hex 00000004; asc ;;
1: len 6; hex 0000000084ab; asc ;;
03:27:03 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=10
max_threads=1024
thread_count=9
connection_count=9
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2248688 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x33acbb0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fa96d316e48 thread_stack 0x40000
/tmp/galera/local1/mysql/sbin/mysqld(my_print_stacktrace+0x2e)[0x808f82]
/tmp/galera/local1/mysql/sbin/mysqld(handle_fatal_signal+0x2ec)[0x6cc260]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7fa994c10cb0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7fa993c42425]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x17b)[0x7fa993c45b8b]
/tmp/galera/local1/mysql/sbin/mysqld[0x9c7afd]
/tmp/galera/local1/mysql/sbin/mysqld[0x9cd11e]
/tmp/galera/local1/mysql/sbin/mysqld[0x9cdc56]
/tmp/galera/local1/mysql/sbin/mysqld[0x9ce45b]
/tmp/galera/local1/mysql/sbin/mysqld[0x8bd8e4]
/tmp/galera/local1/mysql/sbin/mysqld[0x8c6949]
/tmp/galera/local1/mysql/sbin/mysqld[0x87f6be]
/tmp/galera/local1/mysql/sbin/mysqld(_ZN7handler14index_read_mapEPhPKhm16ha_rkey_function+0x4b)[0x6d5bfd]
/tmp/galera/local1/mysql/sbin/mysqld(_ZN14Rows_log_event8find_rowEPK14Relay_log_info+0x448)[0x77add2]
/tmp/galera/local1/mysql/sbin/mysqld(_ZN21Delete_rows_log_event11do_exec_rowEPK14Relay_log_info+0xcf)[0x77bdf7]
/tmp/galera/local1/mysql/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0x843)[0x77097b]
/tmp/galera/local1/mysql/sbin/mysqld(_Z14wsrep_apply_cbPvPKvmjPK14wsrep_trx_meta+0x589)[0x687bc1]
/tmp/galera/local1/galera/lib/libgalera_smm.so(+0x2d1c36)[0x7fa9926f8c36]
/tmp/galera/local1/galera/lib/libgalera_smm.so(+0x2d1f28)[0x7fa9926f8f28]
/tmp/galera/local1/galera/lib/libgalera_smm.so(_ZN6galera13ReplicatorSMM10replay_trxEPNS_9TrxHandleEPv+0x21e)[0x7fa9926fc3dc]
/tmp/galera/local1/galera/lib/libgalera_smm.so(galera_replay_trx+0xc4)[0x7fa99270d00a]
/tmp/galera/local1/mysql/sbin/mysqld(_Z24wsrep_replay_transactionP3THD+0x281)[0x6897d1]
/tmp/galera/local1/mysql/sbin/mysqld[0x5a96c9]
/tmp/galera/local1/mysql/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1145)[0x5aac03]
/tmp/galera/local1/mysql/sbin/mysqld(_Z10do_commandP3THD+0x718)[0x5acea6]
/tmp/galera/local1/mysql/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x11b)[0x655931]
/tmp/galera/local1/mysql/sbin/mysqld(handle_one_connection+0x52)[0x655aed]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fa994c08e9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fa993d003fd]
The problem may relate to fix in: lp:1100496
The anatomy of the issue is as follows:
If a table does not have a primary key, nor any other unique, but has some non-unique keys defined, then DML on the table will result in taking X locks on the non-unique key. Because, the key is non-unique, there will be excessive record X locks on the table, and during parallel replication, some of the excessive X locks, granted for separate transactions, may conflict.
Galera does parallel applying control based on MD5 hash sum on the whole tuple, and it can be that two slave appliers are updating separate rows, but still end up in lock conflict due to these non-unique keys. (if mysql would use full table scan, we would not hit this issue)