Comment 5 for bug 1244573

After the 5.6.22 merge CentOS 5 release build started crashing on the MTR bootstrap shutdown:

Program received signal SIGSEGV, Segmentation fault.
logs_empty_and_mark_files_at_shutdown () at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/storage/innobase/log/log0log.cc:3597
3597 lsn = log_sys->lsn;
(gdb) bt
#0 logs_empty_and_mark_files_at_shutdown () at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/storage/innobase/log/log0log.cc:3597
#1 0x0871f44a in innobase_shutdown_for_mysql () at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/storage/innobase/srv/srv0start.cc:2980
#2 0x08645279 in innobase_end (hton=0x8d5f610, type=HA_PANIC_CLOSE) at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/storage/innobase/handler/ha_innodb.cc:3809
#3 0x081a2fee in ha_finalize_handlerton (plugin=0x8da68f8) at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/handler.cc:641
#4 0x08331bb7 in plugin_deinitialize (plugin=0x8da68f8, ref_check=true) at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/sql_plugin.cc:944
#5 0x08336e7f in reap_plugins () at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/sql_plugin.cc:1023
#6 0x0833833a in plugin_shutdown () at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/sql_plugin.cc:1725
#7 0x08197233 in clean_up (print_message=true) at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/mysqld.cc:1935
#8 0x08197975 in unireg_abort (exit_code=2) at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/mysqld.cc:1821
#9 0x0819b728 in init_server_components () at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/mysqld.cc:4992
#10 0x0819bc7d in mysqld_main (argc=19, argv=0x8d5ee68) at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/mysqld.cc:5611
#11 0x0818e4d2 in main (argc=Cannot access memory at address 0x0
) at /home/jenkins/workspace/percona-server-5.6-trunk/BUILD_TYPE/release/Host/centos5-32/sql/main.cc:25

Disassembly suggests that http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51332 might be relevant here too:

(gdb) x/i $pc
0x86943ec <logs_empty_and_mark_files_at_shutdown()+924>: mov 0x113694(%ebx),%esi

0x086943de <logs_empty_and_mark_files_at_shutdown()+910>: xchg %eax,%ebx
0x086943df <logs_empty_and_mark_files_at_shutdown()+911>: lock cmpxchg8b (%esi)
0x086943e3 <logs_empty_and_mark_files_at_shutdown()+915>: xchg %eax,%ebx
0x086943e4 <logs_empty_and_mark_files_at_shutdown()+916>: mov %eax,-0x38(%ebp)
0x086943e7 <logs_empty_and_mark_files_at_shutdown()+919>: mov %edx,-0x34(%ebp)
0x086943ea <logs_empty_and_mark_files_at_shutdown()+922>: jne 0x86943cd <logs_empty_and_mark_files_at_shutdown()+893>
0x086943ec <logs_empty_and_mark_files_at_shutdown()+924>: mov 0x113694(%ebx),%esi

I haven't worked out the above for a 100% confirmation, but I believe we must switch to GCC 4.4 before any further investigation.