111012 10:56:51 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 111012 10:56:51 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 10:56:51 [Note] Flashcache bypass: disabled 111012 10:56:51 [Note] Flashcache setup error is : ioctl failed 111012 10:56:51 [Warning] You need to use --log-bin to make --binlog-format work. 111012 10:56:51 [Note] Plugin 'FEDERATED' is disabled. 111012 10:56:51 InnoDB: The InnoDB memory heap is disabled 111012 10:56:51 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 10:56:51 InnoDB: Compressed tables use zlib 1.2.3 111012 10:56:51 InnoDB: Using Linux native AIO 111012 10:56:51 InnoDB: Initializing buffer pool, size = 2.0G 111012 10:56:51 InnoDB: Completed initialization of buffer pool 111012 10:56:51 InnoDB: highest supported file format is Barracuda. 111012 10:56:52 InnoDB: Waiting for the background threads to start 111012 10:56:53 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 107985434436 111012 10:56:54 [ERROR] Failed to open the relay log './dbslave1-relay-bin.000001' (relay_log_pos 4) 111012 10:56:54 [ERROR] Could not find target log during relay log initialization 111012 10:56:54 [ERROR] Failed to initialize the master info structure 111012 10:56:54 [Note] Event Scheduler: Loaded 0 events 111012 10:56:54 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 10:58:16 [Note] /usr/sbin/mysqld: Normal shutdown 111012 10:58:16 [Note] Event Scheduler: Purging the queue. 0 events 111012 10:58:16 InnoDB: Starting shutdown... 111012 10:58:18 InnoDB: Shutdown completed; log sequence number 107985434436 111012 10:58:18 [Note] /usr/sbin/mysqld: Shutdown complete 111012 10:58:18 mysqld_safe mysqld from pid file /var/lib/mysql/dbslave1.nextwavemedia.com.pid ended 111012 10:58:24 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 111012 10:58:24 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 10:58:24 [Note] Flashcache bypass: disabled 111012 10:58:24 [Note] Flashcache setup error is : ioctl failed 111012 10:58:24 [Warning] You need to use --log-bin to make --binlog-format work. 111012 10:58:24 [Note] Plugin 'FEDERATED' is disabled. 111012 10:58:24 InnoDB: The InnoDB memory heap is disabled 111012 10:58:24 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 10:58:24 InnoDB: Compressed tables use zlib 1.2.3 111012 10:58:24 InnoDB: Using Linux native AIO 111012 10:58:24 InnoDB: Initializing buffer pool, size = 2.0G 111012 10:58:24 InnoDB: Completed initialization of buffer pool 111012 10:58:24 InnoDB: highest supported file format is Barracuda. 111012 10:58:24 InnoDB: Waiting for the background threads to start 111012 10:58:25 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 107985434436 111012 10:58:25 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000001' position: 4 111012 10:58:25 [Note] Event Scheduler: Loaded 0 events 111012 10:58:25 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 10:58:25 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000028' at position 569529246 111012 14:12:27 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=1 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x17ae01d0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41b1d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:12:28 mysqld_safe Number of processes running now: 0 111012 14:12:28 mysqld_safe mysqld restarted 111012 14:12:28 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:12:28 [Note] Flashcache bypass: disabled 111012 14:12:28 [Note] Flashcache setup error is : ioctl failed 111012 14:12:28 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:12:28 [Note] Plugin 'FEDERATED' is disabled. 111012 14:12:28 InnoDB: The InnoDB memory heap is disabled 111012 14:12:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:12:28 InnoDB: Compressed tables use zlib 1.2.3 111012 14:12:28 InnoDB: Using Linux native AIO 111012 14:12:28 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:12:28 InnoDB: Completed initialization of buffer pool 111012 14:12:28 InnoDB: highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 107994496709 111012 14:12:28 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 107999739392 InnoDB: Doing recovery: scanned up to log sequence number 108004982272 InnoDB: Doing recovery: scanned up to log sequence number 108010225152 InnoDB: Doing recovery: scanned up to log sequence number 108015468032 InnoDB: Doing recovery: scanned up to log sequence number 108020710912 InnoDB: Doing recovery: scanned up to log sequence number 108025953792 InnoDB: Doing recovery: scanned up to log sequence number 108031196672 InnoDB: Doing recovery: scanned up to log sequence number 108036439552 InnoDB: Doing recovery: scanned up to log sequence number 108041682432 InnoDB: Doing recovery: scanned up to log sequence number 108046925312 InnoDB: Doing recovery: scanned up to log sequence number 108052168192 InnoDB: Doing recovery: scanned up to log sequence number 108057411072 InnoDB: Doing recovery: scanned up to log sequence number 108062653952 InnoDB: Doing recovery: scanned up to log sequence number 108067896832 InnoDB: Doing recovery: scanned up to log sequence number 108073139712 InnoDB: Doing recovery: scanned up to log sequence number 108078382592 InnoDB: Doing recovery: scanned up to log sequence number 108083625472 InnoDB: Doing recovery: scanned up to log sequence number 108088868352 InnoDB: Doing recovery: scanned up to log sequence number 108094111232 InnoDB: Doing recovery: scanned up to log sequence number 108099354112 InnoDB: Doing recovery: scanned up to log sequence number 108104596992 InnoDB: Doing recovery: scanned up to log sequence number 108109839872 InnoDB: Doing recovery: scanned up to log sequence number 108115082752 InnoDB: Doing recovery: scanned up to log sequence number 108120325632 InnoDB: Doing recovery: scanned up to log sequence number 108125568512 InnoDB: Doing recovery: scanned up to log sequence number 108130811392 InnoDB: Doing recovery: scanned up to log sequence number 108136054272 InnoDB: Doing recovery: scanned up to log sequence number 108141297152 InnoDB: Doing recovery: scanned up to log sequence number 108146540032 InnoDB: Doing recovery: scanned up to log sequence number 108151782912 InnoDB: Doing recovery: scanned up to log sequence number 108157025792 InnoDB: Doing recovery: scanned up to log sequence number 108162268672 InnoDB: Doing recovery: scanned up to log sequence number 108167511552 InnoDB: Doing recovery: scanned up to log sequence number 108172754432 InnoDB: Doing recovery: scanned up to log sequence number 108177997312 InnoDB: Doing recovery: scanned up to log sequence number 108183240192 InnoDB: Doing recovery: scanned up to log sequence number 108188483072 InnoDB: Doing recovery: scanned up to log sequence number 108193725952 InnoDB: Doing recovery: scanned up to log sequence number 108198968832 InnoDB: Doing recovery: scanned up to log sequence number 108204211712 InnoDB: Doing recovery: scanned up to log sequence number 108209454592 InnoDB: Doing recovery: scanned up to log sequence number 108214697472 InnoDB: Doing recovery: scanned up to log sequence number 108219940352 InnoDB: Doing recovery: scanned up to log sequence number 108225183232 InnoDB: Doing recovery: scanned up to log sequence number 108230426112 InnoDB: Doing recovery: scanned up to log sequence number 108233060148 111012 14:13:01 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:16:22 InnoDB: Waiting for the background threads to start 111012 14:16:23 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233060148 111012 14:16:23 [ERROR] /usr/sbin/mysqld: Table './mysql/user' is marked as crashed and should be repaired 111012 14:16:23 [Warning] Checking table: './mysql/user' 111012 14:16:23 [ERROR] 1 client is using or hasn't closed the table properly 111012 14:16:24 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:16:24 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:16:24 [ERROR] /usr/sbin/mysqld: Table './campaign_manager/traffic_clicks_2011_10' is marked as crashed and should be repaired 111012 14:16:24 [Warning] Checking table: './campaign_manager/traffic_clicks_2011_10' 111012 14:16:24 [ERROR] 1 client is using or hasn't closed the table properly 111012 14:16:25 [Note] Event Scheduler: Loaded 0 events 111012 14:16:25 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:16:25 [ERROR] /usr/sbin/mysqld: Table './campaign_manager/traffic_clicks_2011_09' is marked as crashed and should be repaired 111012 14:16:25 [Warning] Checking table: './campaign_manager/traffic_clicks_2011_09' 111012 14:16:25 [ERROR] 1 client is using or hasn't closed the table properly 111012 14:16:27 [ERROR] /usr/sbin/mysqld: Table './campaign_manager/traffic_clicks_2011_08' is marked as crashed and should be repaired 111012 14:16:27 [Warning] Checking table: './campaign_manager/traffic_clicks_2011_08' 111012 14:16:27 [ERROR] 1 client is using or hasn't closed the table properly 111012 14:16:28 [ERROR] /usr/sbin/mysqld: Table './campaign_manager/traffic_clicks_2011_07' is marked as crashed and should be repaired 111012 14:16:28 [Warning] Checking table: './campaign_manager/traffic_clicks_2011_07' 111012 14:16:28 [ERROR] 1 client is using or hasn't closed the table properly 111012 14:16:30 [ERROR] /usr/sbin/mysqld: Table './campaign_manager/traffic_clicks_2011_06' is marked as crashed and should be repaired 111012 14:16:30 [Warning] Checking table: './campaign_manager/traffic_clicks_2011_06' 111012 14:16:30 [ERROR] 1 client is using or hasn't closed the table properly 111012 14:16:31 [ERROR] Slave SQL: Column 0 of table 'campaign_manager.traffic_clicks_2010_01' cannot be converted from type '' to type 'int(10) unsigned', Error_code: 1677 111012 14:16:31 [Warning] Slave: Table './campaign_manager/traffic_clicks_2011_10' is marked as crashed and should be repaired Error_code: 145 111012 14:16:31 [Warning] Slave: Table 'traffic_clicks_2011_10' is marked as crashed and should be repaired Error_code: 1194 111012 14:16:31 [Warning] Slave: Table './campaign_manager/traffic_clicks_2011_09' is marked as crashed and should be repaired Error_code: 145 111012 14:16:31 [Warning] Slave: Table 'traffic_clicks_2011_09' is marked as crashed and should be repaired Error_code: 1194 111012 14:16:31 [Warning] Slave: Table './campaign_manager/traffic_clicks_2011_08' is marked as crashed and should be repaired Error_code: 145 111012 14:16:31 [Warning] Slave: Table 'traffic_clicks_2011_08' is marked as crashed and should be repaired Error_code: 1194 111012 14:16:31 [Warning] Slave: Table './campaign_manager/traffic_clicks_2011_07' is marked as crashed and should be repaired Error_code: 145 111012 14:16:31 [Warning] Slave: Table 'traffic_clicks_2011_07' is marked as crashed and should be repaired Error_code: 1194 111012 14:16:31 [Warning] Slave: Table './campaign_manager/traffic_clicks_2011_06' is marked as crashed and should be repaired Error_code: 145 111012 14:16:31 [Warning] Slave: Table 'traffic_clicks_2011_06' is marked as crashed and should be repaired Error_code: 1194 111012 14:16:31 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.000068' position 534536256 111012 14:19:15 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:15 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=1 max_threads=100 thread_count=1 connection_count=1 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x14c921f0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4142e0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 7 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:15 mysqld_safe Number of processes running now: 0 111012 14:19:15 mysqld_safe mysqld restarted 111012 14:19:15 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:15 [Note] Flashcache bypass: disabled 111012 14:19:15 [Note] Flashcache setup error is : ioctl failed 111012 14:19:15 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:15 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:15 InnoDB: The InnoDB memory heap is disabled 111012 14:19:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:15 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:15 InnoDB: Using Linux native AIO 111012 14:19:15 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:16 InnoDB: Completed initialization of buffer pool 111012 14:19:16 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:16 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:16 InnoDB: Waiting for the background threads to start 111012 14:19:17 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:17 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:17 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:17 [Note] Event Scheduler: Loaded 0 events 111012 14:19:17 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:17 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x83e9800 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x502800f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:17 mysqld_safe Number of processes running now: 0 111012 14:19:17 mysqld_safe mysqld restarted 111012 14:19:17 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:17 [Note] Flashcache bypass: disabled 111012 14:19:17 [Note] Flashcache setup error is : ioctl failed 111012 14:19:17 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:17 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:17 InnoDB: The InnoDB memory heap is disabled 111012 14:19:17 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:17 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:17 InnoDB: Using Linux native AIO 111012 14:19:17 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:17 InnoDB: Completed initialization of buffer pool 111012 14:19:17 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:17 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:18 InnoDB: Waiting for the background threads to start 111012 14:19:19 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:19 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:19 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:19 [Note] Event Scheduler: Loaded 0 events 111012 14:19:19 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:19 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1ec876a0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41c890f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:19 mysqld_safe Number of processes running now: 0 111012 14:19:19 mysqld_safe mysqld restarted 111012 14:19:19 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:19 [Note] Flashcache bypass: disabled 111012 14:19:19 [Note] Flashcache setup error is : ioctl failed 111012 14:19:19 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:19 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:19 InnoDB: The InnoDB memory heap is disabled 111012 14:19:19 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:19 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:19 InnoDB: Using Linux native AIO 111012 14:19:19 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:19 InnoDB: Completed initialization of buffer pool 111012 14:19:19 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:19 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:19 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:19 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:19 [Note] Event Scheduler: Loaded 0 events 111012 14:19:19 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:19 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:19 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1f2a3110 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5030b0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:19 mysqld_safe Number of processes running now: 0 111012 14:19:19 mysqld_safe mysqld restarted 111012 14:19:19 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:19 [Note] Flashcache bypass: disabled 111012 14:19:19 [Note] Flashcache setup error is : ioctl failed 111012 14:19:19 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:19 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:19 InnoDB: The InnoDB memory heap is disabled 111012 14:19:19 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:19 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:19 InnoDB: Using Linux native AIO 111012 14:19:19 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:20 InnoDB: Completed initialization of buffer pool 111012 14:19:20 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:20 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:20 InnoDB: Waiting for the background threads to start 111012 14:19:21 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:21 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:21 [Note] Event Scheduler: Loaded 0 events 111012 14:19:21 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:21 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:21 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x413130f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:21 mysqld_safe Number of processes running now: 0 111012 14:19:21 mysqld_safe mysqld restarted 111012 14:19:21 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:21 [Note] Flashcache bypass: disabled 111012 14:19:21 [Note] Flashcache setup error is : ioctl failed 111012 14:19:21 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:21 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:21 InnoDB: The InnoDB memory heap is disabled 111012 14:19:21 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:21 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:21 InnoDB: Using Linux native AIO 111012 14:19:21 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:21 InnoDB: Completed initialization of buffer pool 111012 14:19:21 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:21 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:21 InnoDB: Waiting for the background threads to start 111012 14:19:22 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:22 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:22 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:22 [Note] Event Scheduler: Loaded 0 events 111012 14:19:22 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:22 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fde40f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:22 mysqld_safe Number of processes running now: 0 111012 14:19:22 mysqld_safe mysqld restarted 111012 14:19:22 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:22 [Note] Flashcache bypass: disabled 111012 14:19:22 [Note] Flashcache setup error is : ioctl failed 111012 14:19:22 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:22 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:22 InnoDB: The InnoDB memory heap is disabled 111012 14:19:22 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:22 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:22 InnoDB: Using Linux native AIO 111012 14:19:23 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:23 InnoDB: Completed initialization of buffer pool 111012 14:19:23 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:23 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:23 InnoDB: Waiting for the background threads to start 111012 14:19:24 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:24 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:24 [Note] Event Scheduler: Loaded 0 events 111012 14:19:24 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:24 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:24 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xdd98980 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40fc20f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:24 mysqld_safe Number of processes running now: 0 111012 14:19:24 mysqld_safe mysqld restarted 111012 14:19:24 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:24 [Note] Flashcache bypass: disabled 111012 14:19:24 [Note] Flashcache setup error is : ioctl failed 111012 14:19:24 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:24 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:24 InnoDB: The InnoDB memory heap is disabled 111012 14:19:24 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:24 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:24 InnoDB: Using Linux native AIO 111012 14:19:24 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:24 InnoDB: Completed initialization of buffer pool 111012 14:19:24 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:24 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:25 InnoDB: Waiting for the background threads to start 111012 14:19:26 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:26 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:26 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:26 [Note] Event Scheduler: Loaded 0 events 111012 14:19:26 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:26 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x16788fe0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x508140f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:26 mysqld_safe Number of processes running now: 0 111012 14:19:26 mysqld_safe mysqld restarted 111012 14:19:26 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:26 [Note] Flashcache bypass: disabled 111012 14:19:26 [Note] Flashcache setup error is : ioctl failed 111012 14:19:26 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:26 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:26 InnoDB: The InnoDB memory heap is disabled 111012 14:19:26 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:26 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:26 InnoDB: Using Linux native AIO 111012 14:19:26 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:26 InnoDB: Completed initialization of buffer pool 111012 14:19:26 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:26 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:26 InnoDB: Waiting for the background threads to start 111012 14:19:27 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:27 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:27 [Note] Event Scheduler: Loaded 0 events 111012 14:19:27 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:27 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:27 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f5b30f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:27 mysqld_safe Number of processes running now: 0 111012 14:19:27 mysqld_safe mysqld restarted 111012 14:19:27 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:27 [Note] Flashcache bypass: disabled 111012 14:19:27 [Note] Flashcache setup error is : ioctl failed 111012 14:19:27 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:27 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:27 InnoDB: The InnoDB memory heap is disabled 111012 14:19:27 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:27 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:27 InnoDB: Using Linux native AIO 111012 14:19:27 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:28 InnoDB: Completed initialization of buffer pool 111012 14:19:28 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:28 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:28 InnoDB: Waiting for the background threads to start 111012 14:19:29 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:29 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:29 [Note] Event Scheduler: Loaded 0 events 111012 14:19:29 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:29 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:29 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x6306280 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x505610f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:29 mysqld_safe Number of processes running now: 0 111012 14:19:29 mysqld_safe mysqld restarted 111012 14:19:29 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:29 [Note] Flashcache bypass: disabled 111012 14:19:29 [Note] Flashcache setup error is : ioctl failed 111012 14:19:29 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:29 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:29 InnoDB: The InnoDB memory heap is disabled 111012 14:19:29 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:29 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:29 InnoDB: Using Linux native AIO 111012 14:19:29 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:29 InnoDB: Completed initialization of buffer pool 111012 14:19:29 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:29 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:29 InnoDB: Waiting for the background threads to start 111012 14:19:30 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:30 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:30 [Note] Event Scheduler: Loaded 0 events 111012 14:19:30 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:30 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:30 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x409cf0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:30 mysqld_safe Number of processes running now: 0 111012 14:19:30 mysqld_safe mysqld restarted 111012 14:19:31 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:31 [Note] Flashcache bypass: disabled 111012 14:19:31 [Note] Flashcache setup error is : ioctl failed 111012 14:19:31 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:31 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:31 InnoDB: The InnoDB memory heap is disabled 111012 14:19:31 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:31 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:31 InnoDB: Using Linux native AIO 111012 14:19:31 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:31 InnoDB: Completed initialization of buffer pool 111012 14:19:31 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:31 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:31 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:31 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:31 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:31 [Note] Event Scheduler: Loaded 0 events 111012 14:19:31 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:31 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xa35d8f0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x50e530f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:31 mysqld_safe Number of processes running now: 0 111012 14:19:31 mysqld_safe mysqld restarted 111012 14:19:31 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:31 [Note] Flashcache bypass: disabled 111012 14:19:31 [Note] Flashcache setup error is : ioctl failed 111012 14:19:31 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:31 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:31 InnoDB: The InnoDB memory heap is disabled 111012 14:19:31 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:31 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:31 InnoDB: Using Linux native AIO 111012 14:19:31 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:31 InnoDB: Completed initialization of buffer pool 111012 14:19:31 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:31 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:32 InnoDB: Waiting for the background threads to start 111012 14:19:33 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:33 [Note] Failed to execute mysql_file_stat on file '/var/log/mysql/binary/mysql-relay-dbslave1.000119' 111012 14:19:33 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:33 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:33 [Note] Event Scheduler: Loaded 0 events 111012 14:19:33 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:33 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x502400f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:33 mysqld_safe Number of processes running now: 0 111012 14:19:33 mysqld_safe mysqld restarted 111012 14:19:33 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:33 [Note] Flashcache bypass: disabled 111012 14:19:33 [Note] Flashcache setup error is : ioctl failed 111012 14:19:33 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:33 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:33 InnoDB: The InnoDB memory heap is disabled 111012 14:19:33 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:33 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:33 InnoDB: Using Linux native AIO 111012 14:19:33 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:33 InnoDB: Completed initialization of buffer pool 111012 14:19:33 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:33 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:33 InnoDB: Waiting for the background threads to start 111012 14:19:34 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:34 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:34 [Note] Event Scheduler: Loaded 0 events 111012 14:19:34 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:34 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:34 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1fe95a40 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40cba0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:34 mysqld_safe Number of processes running now: 0 111012 14:19:34 mysqld_safe mysqld restarted 111012 14:19:34 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:34 [Note] Flashcache bypass: disabled 111012 14:19:34 [Note] Flashcache setup error is : ioctl failed 111012 14:19:34 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:34 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:34 InnoDB: The InnoDB memory heap is disabled 111012 14:19:34 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:34 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:34 InnoDB: Using Linux native AIO 111012 14:19:34 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:35 InnoDB: Completed initialization of buffer pool 111012 14:19:35 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:35 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:35 InnoDB: Waiting for the background threads to start 111012 14:19:36 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:36 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:36 [Note] Event Scheduler: Loaded 0 events 111012 14:19:36 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:36 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:36 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xbca0340 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4194b0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:36 mysqld_safe Number of processes running now: 0 111012 14:19:36 mysqld_safe mysqld restarted 111012 14:19:36 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:36 [Note] Flashcache bypass: disabled 111012 14:19:36 [Note] Flashcache setup error is : ioctl failed 111012 14:19:36 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:36 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:36 InnoDB: The InnoDB memory heap is disabled 111012 14:19:36 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:36 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:36 InnoDB: Using Linux native AIO 111012 14:19:36 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:36 InnoDB: Completed initialization of buffer pool 111012 14:19:36 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:36 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:36 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:37 [Note] Event Scheduler: Loaded 0 events 111012 14:19:37 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:37 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:37 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:37 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x24382060 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41a650f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:37 mysqld_safe Number of processes running now: 0 111012 14:19:37 mysqld_safe mysqld restarted 111012 14:19:37 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:37 [Note] Flashcache bypass: disabled 111012 14:19:37 [Note] Flashcache setup error is : ioctl failed 111012 14:19:37 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:37 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:37 InnoDB: The InnoDB memory heap is disabled 111012 14:19:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:37 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:37 InnoDB: Using Linux native AIO 111012 14:19:37 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:37 InnoDB: Completed initialization of buffer pool 111012 14:19:37 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:37 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:37 InnoDB: Waiting for the background threads to start 111012 14:19:38 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:38 [Note] Event Scheduler: Loaded 0 events 111012 14:19:38 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:38 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:38 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:38 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x131aad90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41c890f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:38 mysqld_safe Number of processes running now: 0 111012 14:19:38 mysqld_safe mysqld restarted 111012 14:19:38 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:38 [Note] Flashcache bypass: disabled 111012 14:19:38 [Note] Flashcache setup error is : ioctl failed 111012 14:19:38 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:38 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:38 InnoDB: The InnoDB memory heap is disabled 111012 14:19:38 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:38 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:38 InnoDB: Using Linux native AIO 111012 14:19:38 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:38 InnoDB: Completed initialization of buffer pool 111012 14:19:38 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:39 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:39 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:39 [Note] Event Scheduler: Loaded 0 events 111012 14:19:39 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:39 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:39 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:39 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1b438a30 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fa1b0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:39 mysqld_safe Number of processes running now: 0 111012 14:19:39 mysqld_safe mysqld restarted 111012 14:19:39 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:39 [Note] Flashcache bypass: disabled 111012 14:19:39 [Note] Flashcache setup error is : ioctl failed 111012 14:19:39 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:39 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:39 InnoDB: The InnoDB memory heap is disabled 111012 14:19:39 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:39 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:39 InnoDB: Using Linux native AIO 111012 14:19:39 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:39 InnoDB: Completed initialization of buffer pool 111012 14:19:39 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:39 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:39 InnoDB: Waiting for the background threads to start 111012 14:19:40 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:40 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:40 [Note] Event Scheduler: Loaded 0 events 111012 14:19:40 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:40 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:40 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x407380f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:40 mysqld_safe Number of processes running now: 0 111012 14:19:40 mysqld_safe mysqld restarted 111012 14:19:40 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:40 [Note] Flashcache bypass: disabled 111012 14:19:40 [Note] Flashcache setup error is : ioctl failed 111012 14:19:40 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:40 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:40 InnoDB: The InnoDB memory heap is disabled 111012 14:19:40 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:40 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:40 InnoDB: Using Linux native AIO 111012 14:19:40 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:41 InnoDB: Completed initialization of buffer pool 111012 14:19:41 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:41 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:41 InnoDB: Waiting for the background threads to start 111012 14:19:42 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:42 [Note] Event Scheduler: Loaded 0 events 111012 14:19:42 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:42 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:42 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:42 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x22491d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5009f0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:42 mysqld_safe Number of processes running now: 0 111012 14:19:42 mysqld_safe mysqld restarted 111012 14:19:42 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:42 [Note] Flashcache bypass: disabled 111012 14:19:42 [Note] Flashcache setup error is : ioctl failed 111012 14:19:42 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:42 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:42 InnoDB: The InnoDB memory heap is disabled 111012 14:19:42 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:42 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:42 InnoDB: Using Linux native AIO 111012 14:19:42 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:42 InnoDB: Completed initialization of buffer pool 111012 14:19:42 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:42 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:43 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:43 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:43 [Note] Event Scheduler: Loaded 0 events 111012 14:19:43 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:43 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:43 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0010b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41b370f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:43 mysqld_safe Number of processes running now: 0 111012 14:19:43 mysqld_safe mysqld restarted 111012 14:19:43 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:43 [Note] Flashcache bypass: disabled 111012 14:19:43 [Note] Flashcache setup error is : ioctl failed 111012 14:19:43 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:43 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:43 InnoDB: The InnoDB memory heap is disabled 111012 14:19:43 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:43 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:43 InnoDB: Using Linux native AIO 111012 14:19:43 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:43 InnoDB: Completed initialization of buffer pool 111012 14:19:43 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:43 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:43 InnoDB: Waiting for the background threads to start 111012 14:19:44 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:44 [Note] Event Scheduler: Loaded 0 events 111012 14:19:44 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:44 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:44 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:44 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x17a03d80 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f2210f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:44 mysqld_safe Number of processes running now: 0 111012 14:19:44 mysqld_safe mysqld restarted 111012 14:19:44 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:44 [Note] Flashcache bypass: disabled 111012 14:19:44 [Note] Flashcache setup error is : ioctl failed 111012 14:19:44 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:44 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:44 InnoDB: The InnoDB memory heap is disabled 111012 14:19:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:44 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:44 InnoDB: Using Linux native AIO 111012 14:19:44 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:45 InnoDB: Completed initialization of buffer pool 111012 14:19:45 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:45 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:45 InnoDB: Waiting for the background threads to start 111012 14:19:46 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:46 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:46 [Note] Event Scheduler: Loaded 0 events 111012 14:19:46 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:46 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:46 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x16f73280 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x500910f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:46 mysqld_safe Number of processes running now: 0 111012 14:19:46 mysqld_safe mysqld restarted 111012 14:19:46 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:46 [Note] Flashcache bypass: disabled 111012 14:19:46 [Note] Flashcache setup error is : ioctl failed 111012 14:19:46 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:46 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:46 InnoDB: The InnoDB memory heap is disabled 111012 14:19:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:46 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:46 InnoDB: Using Linux native AIO 111012 14:19:46 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:46 InnoDB: Completed initialization of buffer pool 111012 14:19:46 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:46 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:46 InnoDB: Waiting for the background threads to start 111012 14:19:47 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:47 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:47 [Note] Event Scheduler: Loaded 0 events 111012 14:19:47 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:47 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:47 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x130a1850 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f8ac0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:48 mysqld_safe Number of processes running now: 0 111012 14:19:48 mysqld_safe mysqld restarted 111012 14:19:48 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:48 [Note] Flashcache bypass: disabled 111012 14:19:48 [Note] Flashcache setup error is : ioctl failed 111012 14:19:48 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:48 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:48 InnoDB: The InnoDB memory heap is disabled 111012 14:19:48 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:48 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:48 InnoDB: Using Linux native AIO 111012 14:19:48 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:48 InnoDB: Completed initialization of buffer pool 111012 14:19:48 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:48 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:48 InnoDB: Waiting for the background threads to start 111012 14:19:49 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:49 [Note] Event Scheduler: Loaded 0 events 111012 14:19:49 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:49 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:49 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:49 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x92b2d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x408b60f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:49 mysqld_safe Number of processes running now: 0 111012 14:19:49 mysqld_safe mysqld restarted 111012 14:19:49 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:49 [Note] Flashcache bypass: disabled 111012 14:19:49 [Note] Flashcache setup error is : ioctl failed 111012 14:19:49 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:49 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:49 InnoDB: The InnoDB memory heap is disabled 111012 14:19:49 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:49 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:49 InnoDB: Using Linux native AIO 111012 14:19:49 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:50 InnoDB: Completed initialization of buffer pool 111012 14:19:50 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:50 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:50 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:50 [Note] Event Scheduler: Loaded 0 events 111012 14:19:50 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:50 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:50 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:50 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1b54bc10 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5017a0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:50 mysqld_safe Number of processes running now: 0 111012 14:19:50 mysqld_safe mysqld restarted 111012 14:19:50 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:50 [Note] Flashcache bypass: disabled 111012 14:19:50 [Note] Flashcache setup error is : ioctl failed 111012 14:19:50 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:50 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:50 InnoDB: The InnoDB memory heap is disabled 111012 14:19:50 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:50 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:50 InnoDB: Using Linux native AIO 111012 14:19:50 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:50 InnoDB: Completed initialization of buffer pool 111012 14:19:50 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:50 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:50 InnoDB: Waiting for the background threads to start 111012 14:19:51 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:51 [Note] Event Scheduler: Loaded 0 events 111012 14:19:51 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:51 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:51 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:51 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1a785d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x502370f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:51 mysqld_safe Number of processes running now: 0 111012 14:19:51 mysqld_safe mysqld restarted 111012 14:19:51 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:51 [Note] Flashcache bypass: disabled 111012 14:19:51 [Note] Flashcache setup error is : ioctl failed 111012 14:19:51 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:51 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:51 InnoDB: The InnoDB memory heap is disabled 111012 14:19:51 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:51 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:51 InnoDB: Using Linux native AIO 111012 14:19:51 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:52 InnoDB: Completed initialization of buffer pool 111012 14:19:52 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:52 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:52 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:52 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:52 [Note] Event Scheduler: Loaded 0 events 111012 14:19:52 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:52 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c011ff0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... 111012 14:19:52 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 stack_bottom = 0x4ff720f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:52 mysqld_safe Number of processes running now: 0 111012 14:19:52 mysqld_safe mysqld restarted 111012 14:19:52 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:52 [Note] Flashcache bypass: disabled 111012 14:19:52 [Note] Flashcache setup error is : ioctl failed 111012 14:19:52 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:52 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:52 InnoDB: The InnoDB memory heap is disabled 111012 14:19:52 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:52 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:52 InnoDB: Using Linux native AIO 111012 14:19:52 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:52 InnoDB: Completed initialization of buffer pool 111012 14:19:52 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:52 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:53 InnoDB: Waiting for the background threads to start 111012 14:19:54 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:54 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:54 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:54 [Note] Event Scheduler: Loaded 0 events 111012 14:19:54 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:54 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40bde0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:54 mysqld_safe Number of processes running now: 0 111012 14:19:54 mysqld_safe mysqld restarted 111012 14:19:54 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:54 [Note] Flashcache bypass: disabled 111012 14:19:54 [Note] Flashcache setup error is : ioctl failed 111012 14:19:54 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:54 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:54 InnoDB: The InnoDB memory heap is disabled 111012 14:19:54 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:54 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:54 InnoDB: Using Linux native AIO 111012 14:19:54 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:54 InnoDB: Completed initialization of buffer pool 111012 14:19:54 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:54 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:54 InnoDB: Waiting for the background threads to start 111012 14:19:55 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:55 [Note] Event Scheduler: Loaded 0 events 111012 14:19:55 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:55 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:55 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:55 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xa196eb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415990f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:55 mysqld_safe Number of processes running now: 0 111012 14:19:55 mysqld_safe mysqld restarted 111012 14:19:55 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:55 [Note] Flashcache bypass: disabled 111012 14:19:55 [Note] Flashcache setup error is : ioctl failed 111012 14:19:55 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:55 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:55 InnoDB: The InnoDB memory heap is disabled 111012 14:19:55 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:55 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:55 InnoDB: Using Linux native AIO 111012 14:19:55 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:56 InnoDB: Completed initialization of buffer pool 111012 14:19:56 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:56 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:56 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:56 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:56 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:56 [Note] Event Scheduler: Loaded 0 events 111012 14:19:56 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:56 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1d92ce70 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5007b0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:56 mysqld_safe Number of processes running now: 0 111012 14:19:56 mysqld_safe mysqld restarted 111012 14:19:56 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:56 [Note] Flashcache bypass: disabled 111012 14:19:56 [Note] Flashcache setup error is : ioctl failed 111012 14:19:56 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:56 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:56 InnoDB: The InnoDB memory heap is disabled 111012 14:19:56 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:56 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:56 InnoDB: Using Linux native AIO 111012 14:19:56 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:56 InnoDB: Completed initialization of buffer pool 111012 14:19:56 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:56 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:56 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:57 [Note] Event Scheduler: Loaded 0 events 111012 14:19:57 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:57 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:57 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:57 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1ca8fdd0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fd460f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:57 mysqld_safe Number of processes running now: 0 111012 14:19:57 mysqld_safe mysqld restarted 111012 14:19:57 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:57 [Note] Flashcache bypass: disabled 111012 14:19:57 [Note] Flashcache setup error is : ioctl failed 111012 14:19:57 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:57 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:57 InnoDB: The InnoDB memory heap is disabled 111012 14:19:57 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:57 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:57 InnoDB: Using Linux native AIO 111012 14:19:57 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:57 InnoDB: Completed initialization of buffer pool 111012 14:19:57 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:57 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:57 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:57 [Note] Event Scheduler: Loaded 0 events 111012 14:19:57 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:57 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:57 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:57 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x15e42190 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415f10f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:57 mysqld_safe Number of processes running now: 0 111012 14:19:57 mysqld_safe mysqld restarted 111012 14:19:57 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:57 [Note] Flashcache bypass: disabled 111012 14:19:57 [Note] Flashcache setup error is : ioctl failed 111012 14:19:57 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:57 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:57 InnoDB: The InnoDB memory heap is disabled 111012 14:19:57 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:57 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:57 InnoDB: Using Linux native AIO 111012 14:19:57 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:58 InnoDB: Completed initialization of buffer pool 111012 14:19:58 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:58 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:58 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:58 [Note] Event Scheduler: Loaded 0 events 111012 14:19:58 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:58 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:58 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:58 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7c1a670 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fd270f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:58 mysqld_safe Number of processes running now: 0 111012 14:19:58 mysqld_safe mysqld restarted 111012 14:19:58 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:58 [Note] Flashcache bypass: disabled 111012 14:19:58 [Note] Flashcache setup error is : ioctl failed 111012 14:19:58 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:58 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:58 InnoDB: The InnoDB memory heap is disabled 111012 14:19:58 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:58 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:58 InnoDB: Using Linux native AIO 111012 14:19:58 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:58 InnoDB: Completed initialization of buffer pool 111012 14:19:58 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:58 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:58 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:19:58 [Note] Event Scheduler: Loaded 0 events 111012 14:19:58 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:19:58 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:19:58 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:19:58 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x934ed60 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fe010f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:19:58 mysqld_safe Number of processes running now: 0 111012 14:19:58 mysqld_safe mysqld restarted 111012 14:19:58 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:19:58 [Note] Flashcache bypass: disabled 111012 14:19:58 [Note] Flashcache setup error is : ioctl failed 111012 14:19:58 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:19:58 [Note] Plugin 'FEDERATED' is disabled. 111012 14:19:58 InnoDB: The InnoDB memory heap is disabled 111012 14:19:58 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:19:58 InnoDB: Compressed tables use zlib 1.2.3 111012 14:19:58 InnoDB: Using Linux native AIO 111012 14:19:58 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:19:59 InnoDB: Completed initialization of buffer pool 111012 14:19:59 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:19:59 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:19:59 InnoDB: Waiting for the background threads to start 111012 14:20:00 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:00 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:00 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:00 [Note] Event Scheduler: Loaded 0 events 111012 14:20:00 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:00 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1f974ab0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419880f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:00 mysqld_safe Number of processes running now: 0 111012 14:20:00 mysqld_safe mysqld restarted 111012 14:20:00 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:00 [Note] Flashcache bypass: disabled 111012 14:20:00 [Note] Flashcache setup error is : ioctl failed 111012 14:20:00 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:00 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:00 InnoDB: The InnoDB memory heap is disabled 111012 14:20:00 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:00 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:00 InnoDB: Using Linux native AIO 111012 14:20:00 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:00 InnoDB: Completed initialization of buffer pool 111012 14:20:00 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:00 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:01 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:01 [Note] Event Scheduler: Loaded 0 events 111012 14:20:01 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:01 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:01 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:01 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x188feaa0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x50c380f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:01 mysqld_safe Number of processes running now: 0 111012 14:20:01 mysqld_safe mysqld restarted 111012 14:20:01 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:01 [Note] Flashcache bypass: disabled 111012 14:20:01 [Note] Flashcache setup error is : ioctl failed 111012 14:20:01 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:01 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:01 InnoDB: The InnoDB memory heap is disabled 111012 14:20:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:01 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:01 InnoDB: Using Linux native AIO 111012 14:20:01 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:01 InnoDB: Completed initialization of buffer pool 111012 14:20:01 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:01 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:01 InnoDB: Waiting for the background threads to start 111012 14:20:02 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:02 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:02 [Note] Event Scheduler: Loaded 0 events 111012 14:20:02 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:02 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:02 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4193e0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:02 mysqld_safe Number of processes running now: 0 111012 14:20:02 mysqld_safe mysqld restarted 111012 14:20:02 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:02 [Note] Flashcache bypass: disabled 111012 14:20:02 [Note] Flashcache setup error is : ioctl failed 111012 14:20:02 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:02 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:02 InnoDB: The InnoDB memory heap is disabled 111012 14:20:02 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:02 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:02 InnoDB: Using Linux native AIO 111012 14:20:02 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:03 InnoDB: Completed initialization of buffer pool 111012 14:20:03 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:03 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:03 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:03 [Note] Event Scheduler: Loaded 0 events 111012 14:20:03 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:03 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:03 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:03 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1926fae0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x512d80f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:03 mysqld_safe Number of processes running now: 0 111012 14:20:03 mysqld_safe mysqld restarted 111012 14:20:03 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:03 [Note] Flashcache bypass: disabled 111012 14:20:03 [Note] Flashcache setup error is : ioctl failed 111012 14:20:03 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:03 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:03 InnoDB: The InnoDB memory heap is disabled 111012 14:20:03 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:03 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:03 InnoDB: Using Linux native AIO 111012 14:20:03 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:03 InnoDB: Completed initialization of buffer pool 111012 14:20:03 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:03 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:03 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:03 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:03 [Note] Event Scheduler: Loaded 0 events 111012 14:20:03 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:03 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:03 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xd684d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x508210f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:03 mysqld_safe Number of processes running now: 0 111012 14:20:03 mysqld_safe mysqld restarted 111012 14:20:03 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:03 [Note] Flashcache bypass: disabled 111012 14:20:03 [Note] Flashcache setup error is : ioctl failed 111012 14:20:03 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:03 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:03 InnoDB: The InnoDB memory heap is disabled 111012 14:20:03 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:03 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:03 InnoDB: Using Linux native AIO 111012 14:20:03 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:04 InnoDB: Completed initialization of buffer pool 111012 14:20:04 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:04 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:04 InnoDB: Waiting for the background threads to start 111012 14:20:05 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:05 [Note] Event Scheduler: Loaded 0 events 111012 14:20:05 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:05 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:05 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:05 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c008750 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x413a90f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:05 mysqld_safe Number of processes running now: 0 111012 14:20:05 mysqld_safe mysqld restarted 111012 14:20:05 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:05 [Note] Flashcache bypass: disabled 111012 14:20:05 [Note] Flashcache setup error is : ioctl failed 111012 14:20:05 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:05 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:05 InnoDB: The InnoDB memory heap is disabled 111012 14:20:05 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:05 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:05 InnoDB: Using Linux native AIO 111012 14:20:05 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:05 InnoDB: Completed initialization of buffer pool 111012 14:20:05 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:05 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:06 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:06 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:06 [Note] Event Scheduler: Loaded 0 events 111012 14:20:06 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:06 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:06 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1512f860 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fda90f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:06 mysqld_safe Number of processes running now: 0 111012 14:20:06 mysqld_safe mysqld restarted 111012 14:20:06 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:06 [Note] Flashcache bypass: disabled 111012 14:20:06 [Note] Flashcache setup error is : ioctl failed 111012 14:20:06 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:06 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:06 InnoDB: The InnoDB memory heap is disabled 111012 14:20:06 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:06 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:06 InnoDB: Using Linux native AIO 111012 14:20:06 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:06 InnoDB: Completed initialization of buffer pool 111012 14:20:06 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:06 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:06 InnoDB: Waiting for the background threads to start 111012 14:20:07 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:07 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:07 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:07 [Note] Event Scheduler: Loaded 0 events 111012 14:20:07 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:07 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fcb80f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:07 mysqld_safe Number of processes running now: 0 111012 14:20:07 mysqld_safe mysqld restarted 111012 14:20:07 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:07 [Note] Flashcache bypass: disabled 111012 14:20:07 [Note] Flashcache setup error is : ioctl failed 111012 14:20:07 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:07 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:07 InnoDB: The InnoDB memory heap is disabled 111012 14:20:07 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:07 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:07 InnoDB: Using Linux native AIO 111012 14:20:07 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:08 InnoDB: Completed initialization of buffer pool 111012 14:20:08 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:08 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:08 InnoDB: Waiting for the background threads to start 111012 14:20:09 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:09 [Note] Event Scheduler: Loaded 0 events 111012 14:20:09 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:09 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:09 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:09 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2495ceb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41def0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:09 mysqld_safe Number of processes running now: 0 111012 14:20:09 mysqld_safe mysqld restarted 111012 14:20:09 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:09 [Note] Flashcache bypass: disabled 111012 14:20:09 [Note] Flashcache setup error is : ioctl failed 111012 14:20:09 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:09 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:09 InnoDB: The InnoDB memory heap is disabled 111012 14:20:09 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:09 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:09 InnoDB: Using Linux native AIO 111012 14:20:09 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:09 InnoDB: Completed initialization of buffer pool 111012 14:20:09 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:09 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:09 InnoDB: Waiting for the background threads to start 111012 14:20:10 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:10 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:10 [Note] Event Scheduler: Loaded 0 events 111012 14:20:10 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:10 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:10 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x8cc4c10 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x405060f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:10 mysqld_safe Number of processes running now: 0 111012 14:20:10 mysqld_safe mysqld restarted 111012 14:20:10 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:10 [Note] Flashcache bypass: disabled 111012 14:20:10 [Note] Flashcache setup error is : ioctl failed 111012 14:20:10 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:10 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:10 InnoDB: The InnoDB memory heap is disabled 111012 14:20:10 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:10 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:10 InnoDB: Using Linux native AIO 111012 14:20:10 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:11 InnoDB: Completed initialization of buffer pool 111012 14:20:11 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:11 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:11 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:11 [Note] Event Scheduler: Loaded 0 events 111012 14:20:11 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:11 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:11 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:11 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x17ed3b30 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41e4a0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:11 mysqld_safe Number of processes running now: 0 111012 14:20:11 mysqld_safe mysqld restarted 111012 14:20:11 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:11 [Note] Flashcache bypass: disabled 111012 14:20:11 [Note] Flashcache setup error is : ioctl failed 111012 14:20:11 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:11 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:11 InnoDB: The InnoDB memory heap is disabled 111012 14:20:11 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:11 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:11 InnoDB: Using Linux native AIO 111012 14:20:11 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:11 InnoDB: Completed initialization of buffer pool 111012 14:20:11 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:11 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:12 InnoDB: Waiting for the background threads to start 111012 14:20:13 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:13 [Note] Event Scheduler: Loaded 0 events 111012 14:20:13 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:13 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:13 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:13 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x10f9cd90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fcd70f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:13 mysqld_safe Number of processes running now: 0 111012 14:20:13 mysqld_safe mysqld restarted 111012 14:20:13 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:13 [Note] Flashcache bypass: disabled 111012 14:20:13 [Note] Flashcache setup error is : ioctl failed 111012 14:20:13 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:13 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:13 InnoDB: The InnoDB memory heap is disabled 111012 14:20:13 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:13 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:13 InnoDB: Using Linux native AIO 111012 14:20:13 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:13 InnoDB: Completed initialization of buffer pool 111012 14:20:13 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:13 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:13 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:13 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:13 [Note] Event Scheduler: Loaded 0 events 111012 14:20:13 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:13 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:13 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x83979b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x501b90f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:13 mysqld_safe Number of processes running now: 0 111012 14:20:13 mysqld_safe mysqld restarted 111012 14:20:13 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:13 [Note] Flashcache bypass: disabled 111012 14:20:13 [Note] Flashcache setup error is : ioctl failed 111012 14:20:13 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:13 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:13 InnoDB: The InnoDB memory heap is disabled 111012 14:20:13 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:13 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:13 InnoDB: Using Linux native AIO 111012 14:20:13 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:14 InnoDB: Completed initialization of buffer pool 111012 14:20:14 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:14 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:14 InnoDB: Waiting for the background threads to start 111012 14:20:15 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:15 [Note] Event Scheduler: Loaded 0 events 111012 14:20:15 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:15 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:15 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:15 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x10426d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f63e0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:15 mysqld_safe Number of processes running now: 0 111012 14:20:15 mysqld_safe mysqld restarted 111012 14:20:15 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:15 [Note] Flashcache bypass: disabled 111012 14:20:15 [Note] Flashcache setup error is : ioctl failed 111012 14:20:15 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:15 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:15 InnoDB: The InnoDB memory heap is disabled 111012 14:20:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:15 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:15 InnoDB: Using Linux native AIO 111012 14:20:15 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:15 InnoDB: Completed initialization of buffer pool 111012 14:20:15 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:15 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:15 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:15 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:15 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:15 [Note] Event Scheduler: Loaded 0 events 111012 14:20:15 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:15 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x10e308e0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x407f00f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:15 mysqld_safe Number of processes running now: 0 111012 14:20:15 mysqld_safe mysqld restarted 111012 14:20:15 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:15 [Note] Flashcache bypass: disabled 111012 14:20:15 [Note] Flashcache setup error is : ioctl failed 111012 14:20:15 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:15 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:15 InnoDB: The InnoDB memory heap is disabled 111012 14:20:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:15 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:15 InnoDB: Using Linux native AIO 111012 14:20:16 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:16 InnoDB: Completed initialization of buffer pool 111012 14:20:16 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:16 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:16 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:16 [Note] Event Scheduler: Loaded 0 events 111012 14:20:16 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:16 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:16 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:16 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xcd09280 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x504d40f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:16 mysqld_safe Number of processes running now: 0 111012 14:20:16 mysqld_safe mysqld restarted 111012 14:20:16 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:16 [Note] Flashcache bypass: disabled 111012 14:20:16 [Note] Flashcache setup error is : ioctl failed 111012 14:20:16 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:16 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:16 InnoDB: The InnoDB memory heap is disabled 111012 14:20:16 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:16 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:16 InnoDB: Using Linux native AIO 111012 14:20:16 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:16 InnoDB: Completed initialization of buffer pool 111012 14:20:16 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:16 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:17 InnoDB: Waiting for the background threads to start 111012 14:20:18 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:18 [Note] Event Scheduler: Loaded 0 events 111012 14:20:18 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:18 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:18 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:18 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x202241b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415280f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:18 mysqld_safe Number of processes running now: 0 111012 14:20:18 mysqld_safe mysqld restarted 111012 14:20:18 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:18 [Note] Flashcache bypass: disabled 111012 14:20:18 [Note] Flashcache setup error is : ioctl failed 111012 14:20:18 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:18 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:18 InnoDB: The InnoDB memory heap is disabled 111012 14:20:18 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:18 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:18 InnoDB: Using Linux native AIO 111012 14:20:18 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:18 InnoDB: Completed initialization of buffer pool 111012 14:20:18 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:18 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:18 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:18 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:18 [Note] Event Scheduler: Loaded 0 events 111012 14:20:18 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:18 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:18 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xb1f48a0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x412a50f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:18 mysqld_safe Number of processes running now: 0 111012 14:20:18 mysqld_safe mysqld restarted 111012 14:20:18 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:18 [Note] Flashcache bypass: disabled 111012 14:20:18 [Note] Flashcache setup error is : ioctl failed 111012 14:20:18 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:18 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:18 InnoDB: The InnoDB memory heap is disabled 111012 14:20:18 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:18 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:18 InnoDB: Using Linux native AIO 111012 14:20:18 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:19 InnoDB: Completed initialization of buffer pool 111012 14:20:19 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:19 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:19 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:19 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:19 [Note] Event Scheduler: Loaded 0 events 111012 14:20:19 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:19 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:19 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab40000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x411f50f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:19 mysqld_safe Number of processes running now: 0 111012 14:20:19 mysqld_safe mysqld restarted 111012 14:20:19 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:19 [Note] Flashcache bypass: disabled 111012 14:20:19 [Note] Flashcache setup error is : ioctl failed 111012 14:20:19 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:19 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:19 InnoDB: The InnoDB memory heap is disabled 111012 14:20:19 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:19 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:19 InnoDB: Using Linux native AIO 111012 14:20:19 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:19 InnoDB: Completed initialization of buffer pool 111012 14:20:19 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:19 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:19 InnoDB: Waiting for the background threads to start 111012 14:20:20 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:20 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:20 [Note] Event Scheduler: Loaded 0 events 111012 14:20:20 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:20 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:20 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41aee0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:20 mysqld_safe Number of processes running now: 0 111012 14:20:20 mysqld_safe mysqld restarted 111012 14:20:20 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:20 [Note] Flashcache bypass: disabled 111012 14:20:20 [Note] Flashcache setup error is : ioctl failed 111012 14:20:20 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:20 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:20 InnoDB: The InnoDB memory heap is disabled 111012 14:20:20 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:20 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:20 InnoDB: Using Linux native AIO 111012 14:20:20 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:21 InnoDB: Completed initialization of buffer pool 111012 14:20:21 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:21 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:21 InnoDB: Waiting for the background threads to start 111012 14:20:22 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:22 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:22 [Note] Event Scheduler: Loaded 0 events 111012 14:20:22 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:22 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:22 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x195a42e0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40ea90f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:22 mysqld_safe Number of processes running now: 0 111012 14:20:22 mysqld_safe mysqld restarted 111012 14:20:22 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:22 [Note] Flashcache bypass: disabled 111012 14:20:22 [Note] Flashcache setup error is : ioctl failed 111012 14:20:22 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:22 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:22 InnoDB: The InnoDB memory heap is disabled 111012 14:20:22 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:22 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:22 InnoDB: Using Linux native AIO 111012 14:20:22 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:22 InnoDB: Completed initialization of buffer pool 111012 14:20:22 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:22 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:23 InnoDB: Waiting for the background threads to start 111012 14:20:24 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:24 [Note] Event Scheduler: Loaded 0 events 111012 14:20:24 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:24 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:24 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:24 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x68a1d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x413c70f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:24 mysqld_safe Number of processes running now: 0 111012 14:20:24 mysqld_safe mysqld restarted 111012 14:20:24 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:24 [Note] Flashcache bypass: disabled 111012 14:20:24 [Note] Flashcache setup error is : ioctl failed 111012 14:20:24 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:24 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:24 InnoDB: The InnoDB memory heap is disabled 111012 14:20:24 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:24 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:24 InnoDB: Using Linux native AIO 111012 14:20:24 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:24 InnoDB: Completed initialization of buffer pool 111012 14:20:24 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:24 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:24 InnoDB: Waiting for the background threads to start 111012 14:20:25 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:25 [Note] Event Scheduler: Loaded 0 events 111012 14:20:25 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:25 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:25 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:25 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415e90f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:25 mysqld_safe Number of processes running now: 0 111012 14:20:25 mysqld_safe mysqld restarted 111012 14:20:25 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:26 [Note] Flashcache bypass: disabled 111012 14:20:26 [Note] Flashcache setup error is : ioctl failed 111012 14:20:26 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:26 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:26 InnoDB: The InnoDB memory heap is disabled 111012 14:20:26 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:26 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:26 InnoDB: Using Linux native AIO 111012 14:20:26 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:26 InnoDB: Completed initialization of buffer pool 111012 14:20:26 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:26 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:26 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:26 [Note] Event Scheduler: Loaded 0 events 111012 14:20:26 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:26 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:26 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:26 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x199d0b00 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415040f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:26 mysqld_safe Number of processes running now: 0 111012 14:20:26 mysqld_safe mysqld restarted 111012 14:20:26 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:26 [Note] Flashcache bypass: disabled 111012 14:20:26 [Note] Flashcache setup error is : ioctl failed 111012 14:20:26 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:26 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:26 InnoDB: The InnoDB memory heap is disabled 111012 14:20:26 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:26 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:26 InnoDB: Using Linux native AIO 111012 14:20:26 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:26 InnoDB: Completed initialization of buffer pool 111012 14:20:26 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:26 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:27 InnoDB: Waiting for the background threads to start 111012 14:20:28 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:28 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:28 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:28 [Note] Event Scheduler: Loaded 0 events 111012 14:20:28 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:28 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4194d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:28 mysqld_safe Number of processes running now: 0 111012 14:20:28 mysqld_safe mysqld restarted 111012 14:20:28 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:28 [Note] Flashcache bypass: disabled 111012 14:20:28 [Note] Flashcache setup error is : ioctl failed 111012 14:20:28 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:28 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:28 InnoDB: The InnoDB memory heap is disabled 111012 14:20:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:28 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:28 InnoDB: Using Linux native AIO 111012 14:20:28 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:28 InnoDB: Completed initialization of buffer pool 111012 14:20:28 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:28 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:28 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:28 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:28 [Note] Event Scheduler: Loaded 0 events 111012 14:20:28 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:28 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:28 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1937f490 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415bb0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:28 mysqld_safe Number of processes running now: 0 111012 14:20:28 mysqld_safe mysqld restarted 111012 14:20:28 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:28 [Note] Flashcache bypass: disabled 111012 14:20:28 [Note] Flashcache setup error is : ioctl failed 111012 14:20:28 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:28 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:28 InnoDB: The InnoDB memory heap is disabled 111012 14:20:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:28 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:28 InnoDB: Using Linux native AIO 111012 14:20:28 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:29 InnoDB: Completed initialization of buffer pool 111012 14:20:29 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:29 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:29 InnoDB: Waiting for the background threads to start 111012 14:20:30 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:30 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:30 [Note] Event Scheduler: Loaded 0 events 111012 14:20:30 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:30 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:30 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x18446800 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40e840f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:30 mysqld_safe Number of processes running now: 0 111012 14:20:30 mysqld_safe mysqld restarted 111012 14:20:30 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:30 [Note] Flashcache bypass: disabled 111012 14:20:30 [Note] Flashcache setup error is : ioctl failed 111012 14:20:30 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:30 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:30 InnoDB: The InnoDB memory heap is disabled 111012 14:20:30 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:30 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:30 InnoDB: Using Linux native AIO 111012 14:20:30 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:30 InnoDB: Completed initialization of buffer pool 111012 14:20:30 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:30 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:30 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:30 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:30 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:30 [Note] Event Scheduler: Loaded 0 events 111012 14:20:30 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:30 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xf674cb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415ed0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:30 mysqld_safe Number of processes running now: 0 111012 14:20:30 mysqld_safe mysqld restarted 111012 14:20:30 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:30 [Note] Flashcache bypass: disabled 111012 14:20:30 [Note] Flashcache setup error is : ioctl failed 111012 14:20:30 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:30 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:30 InnoDB: The InnoDB memory heap is disabled 111012 14:20:30 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:30 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:30 InnoDB: Using Linux native AIO 111012 14:20:31 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:31 InnoDB: Completed initialization of buffer pool 111012 14:20:31 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:31 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:31 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:31 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:31 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:31 [Note] Event Scheduler: Loaded 0 events 111012 14:20:31 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:31 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c008fd0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x50a180f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:31 mysqld_safe Number of processes running now: 0 111012 14:20:31 mysqld_safe mysqld restarted 111012 14:20:31 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:31 [Note] Flashcache bypass: disabled 111012 14:20:31 [Note] Flashcache setup error is : ioctl failed 111012 14:20:31 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:31 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:31 InnoDB: The InnoDB memory heap is disabled 111012 14:20:31 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:31 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:31 InnoDB: Using Linux native AIO 111012 14:20:31 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:31 InnoDB: Completed initialization of buffer pool 111012 14:20:31 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:31 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:32 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:32 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:32 [Note] Event Scheduler: Loaded 0 events 111012 14:20:32 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:32 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:32 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x14bf6a10 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5025b0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:32 mysqld_safe Number of processes running now: 0 111012 14:20:32 mysqld_safe mysqld restarted 111012 14:20:32 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:32 [Note] Flashcache bypass: disabled 111012 14:20:32 [Note] Flashcache setup error is : ioctl failed 111012 14:20:32 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:32 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:32 InnoDB: The InnoDB memory heap is disabled 111012 14:20:32 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:32 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:32 InnoDB: Using Linux native AIO 111012 14:20:32 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:32 InnoDB: Completed initialization of buffer pool 111012 14:20:32 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:32 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:32 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:32 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:32 [Note] Event Scheduler: Loaded 0 events 111012 14:20:32 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:32 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:32 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0010b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40b060f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:32 mysqld_safe Number of processes running now: 0 111012 14:20:32 mysqld_safe mysqld restarted 111012 14:20:32 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:32 [Note] Flashcache bypass: disabled 111012 14:20:32 [Note] Flashcache setup error is : ioctl failed 111012 14:20:32 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:32 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:32 InnoDB: The InnoDB memory heap is disabled 111012 14:20:32 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:32 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:32 InnoDB: Using Linux native AIO 111012 14:20:32 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:33 InnoDB: Completed initialization of buffer pool 111012 14:20:33 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:33 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:33 InnoDB: Waiting for the background threads to start 111012 14:20:34 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:34 [Note] Event Scheduler: Loaded 0 events 111012 14:20:34 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:34 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:34 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:34 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x189991b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419bc0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:34 mysqld_safe Number of processes running now: 0 111012 14:20:34 mysqld_safe mysqld restarted 111012 14:20:34 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:34 [Note] Flashcache bypass: disabled 111012 14:20:34 [Note] Flashcache setup error is : ioctl failed 111012 14:20:34 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:34 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:34 InnoDB: The InnoDB memory heap is disabled 111012 14:20:34 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:34 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:34 InnoDB: Using Linux native AIO 111012 14:20:34 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:34 InnoDB: Completed initialization of buffer pool 111012 14:20:34 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:34 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:34 InnoDB: Waiting for the background threads to start 111012 14:20:35 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:35 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:35 [Note] Event Scheduler: Loaded 0 events 111012 14:20:35 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:35 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:35 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x23b1c820 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x412a60f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:35 mysqld_safe Number of processes running now: 0 111012 14:20:35 mysqld_safe mysqld restarted 111012 14:20:35 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:35 [Note] Flashcache bypass: disabled 111012 14:20:35 [Note] Flashcache setup error is : ioctl failed 111012 14:20:35 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:35 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:35 InnoDB: The InnoDB memory heap is disabled 111012 14:20:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:35 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:35 InnoDB: Using Linux native AIO 111012 14:20:36 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:36 InnoDB: Completed initialization of buffer pool 111012 14:20:36 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:36 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:36 InnoDB: Waiting for the background threads to start 111012 14:20:37 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:37 [Note] Event Scheduler: Loaded 0 events 111012 14:20:37 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:37 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:37 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:37 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x15dd3d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f66e0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:37 mysqld_safe Number of processes running now: 0 111012 14:20:37 mysqld_safe mysqld restarted 111012 14:20:37 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:37 [Note] Flashcache bypass: disabled 111012 14:20:37 [Note] Flashcache setup error is : ioctl failed 111012 14:20:37 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:37 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:37 InnoDB: The InnoDB memory heap is disabled 111012 14:20:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:37 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:37 InnoDB: Using Linux native AIO 111012 14:20:37 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:37 InnoDB: Completed initialization of buffer pool 111012 14:20:37 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:37 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:38 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:38 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:38 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:38 [Note] Event Scheduler: Loaded 0 events 111012 14:20:38 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:38 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2027d6c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x505550f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:38 mysqld_safe Number of processes running now: 0 111012 14:20:38 mysqld_safe mysqld restarted 111012 14:20:38 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:38 [Note] Flashcache bypass: disabled 111012 14:20:38 [Note] Flashcache setup error is : ioctl failed 111012 14:20:38 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:38 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:38 InnoDB: The InnoDB memory heap is disabled 111012 14:20:38 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:38 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:38 InnoDB: Using Linux native AIO 111012 14:20:38 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:38 InnoDB: Completed initialization of buffer pool 111012 14:20:38 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:38 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:38 InnoDB: Waiting for the background threads to start 111012 14:20:39 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:39 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:39 [Note] Event Scheduler: Loaded 0 events 111012 14:20:39 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:39 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:39 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x500f90f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:39 mysqld_safe Number of processes running now: 0 111012 14:20:39 mysqld_safe mysqld restarted 111012 14:20:39 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:39 [Note] Flashcache bypass: disabled 111012 14:20:39 [Note] Flashcache setup error is : ioctl failed 111012 14:20:39 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:39 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:39 InnoDB: The InnoDB memory heap is disabled 111012 14:20:39 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:39 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:39 InnoDB: Using Linux native AIO 111012 14:20:39 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:40 InnoDB: Completed initialization of buffer pool 111012 14:20:40 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:40 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:40 InnoDB: Waiting for the background threads to start 111012 14:20:41 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:41 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:41 [Note] Event Scheduler: Loaded 0 events 111012 14:20:41 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:41 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:41 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x15bc6eb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419c20f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:41 mysqld_safe Number of processes running now: 0 111012 14:20:41 mysqld_safe mysqld restarted 111012 14:20:41 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:41 [Note] Flashcache bypass: disabled 111012 14:20:41 [Note] Flashcache setup error is : ioctl failed 111012 14:20:41 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:41 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:41 InnoDB: The InnoDB memory heap is disabled 111012 14:20:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:41 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:41 InnoDB: Using Linux native AIO 111012 14:20:41 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:41 InnoDB: Completed initialization of buffer pool 111012 14:20:41 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:41 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:41 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:41 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:41 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:41 [Note] Event Scheduler: Loaded 0 events 111012 14:20:41 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:41 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1ba686b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x501dc0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:41 mysqld_safe Number of processes running now: 0 111012 14:20:41 mysqld_safe mysqld restarted 111012 14:20:41 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:41 [Note] Flashcache bypass: disabled 111012 14:20:41 [Note] Flashcache setup error is : ioctl failed 111012 14:20:41 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:41 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:41 InnoDB: The InnoDB memory heap is disabled 111012 14:20:41 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:41 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:41 InnoDB: Using Linux native AIO 111012 14:20:41 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:42 InnoDB: Completed initialization of buffer pool 111012 14:20:42 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:42 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:42 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:42 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:42 [Note] Event Scheduler: Loaded 0 events 111012 14:20:42 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:42 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:42 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1e8fd6b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x508fd0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:42 mysqld_safe Number of processes running now: 0 111012 14:20:42 mysqld_safe mysqld restarted 111012 14:20:42 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:42 [Note] Flashcache bypass: disabled 111012 14:20:42 [Note] Flashcache setup error is : ioctl failed 111012 14:20:42 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:42 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:42 InnoDB: The InnoDB memory heap is disabled 111012 14:20:42 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:42 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:42 InnoDB: Using Linux native AIO 111012 14:20:42 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:42 InnoDB: Completed initialization of buffer pool 111012 14:20:42 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:42 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:43 InnoDB: Waiting for the background threads to start 111012 14:20:44 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:44 [Note] Event Scheduler: Loaded 0 events 111012 14:20:44 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:44 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:44 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:44 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x138dad90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4133b0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:44 mysqld_safe Number of processes running now: 0 111012 14:20:44 mysqld_safe mysqld restarted 111012 14:20:44 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:44 [Note] Flashcache bypass: disabled 111012 14:20:44 [Note] Flashcache setup error is : ioctl failed 111012 14:20:44 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:44 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:44 InnoDB: The InnoDB memory heap is disabled 111012 14:20:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:44 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:44 InnoDB: Using Linux native AIO 111012 14:20:44 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:44 InnoDB: Completed initialization of buffer pool 111012 14:20:44 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:44 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:44 InnoDB: Waiting for the background threads to start 111012 14:20:45 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:45 [Note] Event Scheduler: Loaded 0 events 111012 14:20:45 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:45 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:45 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:45 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x20b59d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fbe30f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:45 mysqld_safe Number of processes running now: 0 111012 14:20:45 mysqld_safe mysqld restarted 111012 14:20:45 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:45 [Note] Flashcache bypass: disabled 111012 14:20:45 [Note] Flashcache setup error is : ioctl failed 111012 14:20:45 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:45 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:45 InnoDB: The InnoDB memory heap is disabled 111012 14:20:45 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:45 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:45 InnoDB: Using Linux native AIO 111012 14:20:45 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:45 InnoDB: Completed initialization of buffer pool 111012 14:20:46 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:46 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:46 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:46 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:46 [Note] Event Scheduler: Loaded 0 events 111012 14:20:46 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:46 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:46 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1a67a0c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x418fc0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:46 mysqld_safe Number of processes running now: 0 111012 14:20:46 mysqld_safe mysqld restarted 111012 14:20:46 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:46 [Note] Flashcache bypass: disabled 111012 14:20:46 [Note] Flashcache setup error is : ioctl failed 111012 14:20:46 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:46 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:46 InnoDB: The InnoDB memory heap is disabled 111012 14:20:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:46 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:46 InnoDB: Using Linux native AIO 111012 14:20:46 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:46 InnoDB: Completed initialization of buffer pool 111012 14:20:46 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:46 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:46 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:46 [Note] Event Scheduler: Loaded 0 events 111012 14:20:46 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:46 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:46 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:46 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xa6fd7a0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4129d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:46 mysqld_safe Number of processes running now: 0 111012 14:20:46 mysqld_safe mysqld restarted 111012 14:20:46 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:46 [Note] Flashcache bypass: disabled 111012 14:20:46 [Note] Flashcache setup error is : ioctl failed 111012 14:20:46 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:46 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:46 InnoDB: The InnoDB memory heap is disabled 111012 14:20:46 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:46 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:46 InnoDB: Using Linux native AIO 111012 14:20:46 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:47 InnoDB: Completed initialization of buffer pool 111012 14:20:47 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:47 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:47 InnoDB: Waiting for the background threads to start 111012 14:20:48 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:48 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:48 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:48 [Note] Event Scheduler: Loaded 0 events 111012 14:20:48 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:48 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2119b1f0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x503100f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:48 mysqld_safe Number of processes running now: 0 111012 14:20:48 mysqld_safe mysqld restarted 111012 14:20:48 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:48 [Note] Flashcache bypass: disabled 111012 14:20:48 [Note] Flashcache setup error is : ioctl failed 111012 14:20:48 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:48 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:48 InnoDB: The InnoDB memory heap is disabled 111012 14:20:48 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:48 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:48 InnoDB: Using Linux native AIO 111012 14:20:48 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:48 InnoDB: Completed initialization of buffer pool 111012 14:20:48 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:48 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:49 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:49 [Note] Event Scheduler: Loaded 0 events 111012 14:20:49 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:49 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:49 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:49 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x8676a20 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fd600f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:49 mysqld_safe Number of processes running now: 0 111012 14:20:49 mysqld_safe mysqld restarted 111012 14:20:49 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:49 [Note] Flashcache bypass: disabled 111012 14:20:49 [Note] Flashcache setup error is : ioctl failed 111012 14:20:49 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:49 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:49 InnoDB: The InnoDB memory heap is disabled 111012 14:20:49 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:49 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:49 InnoDB: Using Linux native AIO 111012 14:20:49 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:49 InnoDB: Completed initialization of buffer pool 111012 14:20:49 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:49 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:49 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:49 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:49 [Note] Event Scheduler: Loaded 0 events 111012 14:20:49 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:49 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:49 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xc8b9880 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x50f530f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:49 mysqld_safe Number of processes running now: 0 111012 14:20:49 mysqld_safe mysqld restarted 111012 14:20:49 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:49 [Note] Flashcache bypass: disabled 111012 14:20:49 [Note] Flashcache setup error is : ioctl failed 111012 14:20:49 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:49 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:49 InnoDB: The InnoDB memory heap is disabled 111012 14:20:49 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:49 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:49 InnoDB: Using Linux native AIO 111012 14:20:49 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:49 InnoDB: Completed initialization of buffer pool 111012 14:20:50 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:50 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:50 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:50 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:50 [Note] Event Scheduler: Loaded 0 events 111012 14:20:50 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:50 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:50 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x16a31090 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4ff170f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:50 mysqld_safe Number of processes running now: 0 111012 14:20:50 mysqld_safe mysqld restarted 111012 14:20:50 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:50 [Note] Flashcache bypass: disabled 111012 14:20:50 [Note] Flashcache setup error is : ioctl failed 111012 14:20:50 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:50 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:50 InnoDB: The InnoDB memory heap is disabled 111012 14:20:50 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:50 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:50 InnoDB: Using Linux native AIO 111012 14:20:50 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:50 InnoDB: Completed initialization of buffer pool 111012 14:20:50 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:50 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:50 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:50 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:50 [Note] Event Scheduler: Loaded 0 events 111012 14:20:50 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:50 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:50 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab40000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419b40f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:50 mysqld_safe Number of processes running now: 0 111012 14:20:50 mysqld_safe mysqld restarted 111012 14:20:50 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:50 [Note] Flashcache bypass: disabled 111012 14:20:50 [Note] Flashcache setup error is : ioctl failed 111012 14:20:50 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:50 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:50 InnoDB: The InnoDB memory heap is disabled 111012 14:20:50 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:50 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:50 InnoDB: Using Linux native AIO 111012 14:20:50 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:51 InnoDB: Completed initialization of buffer pool 111012 14:20:51 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:51 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:51 InnoDB: Waiting for the background threads to start 111012 14:20:52 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:52 [Note] Event Scheduler: Loaded 0 events 111012 14:20:52 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:52 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:52 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:52 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xd0561b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f5590f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:52 mysqld_safe Number of processes running now: 0 111012 14:20:52 mysqld_safe mysqld restarted 111012 14:20:52 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:52 [Note] Flashcache bypass: disabled 111012 14:20:52 [Note] Flashcache setup error is : ioctl failed 111012 14:20:52 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:52 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:52 InnoDB: The InnoDB memory heap is disabled 111012 14:20:52 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:52 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:52 InnoDB: Using Linux native AIO 111012 14:20:52 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:52 InnoDB: Completed initialization of buffer pool 111012 14:20:52 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:52 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:53 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:53 [Note] Event Scheduler: Loaded 0 events 111012 14:20:53 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:53 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:53 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:53 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xa7e7b10 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40e030f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:53 mysqld_safe Number of processes running now: 0 111012 14:20:53 mysqld_safe mysqld restarted 111012 14:20:53 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:53 [Note] Flashcache bypass: disabled 111012 14:20:53 [Note] Flashcache setup error is : ioctl failed 111012 14:20:53 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:53 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:53 InnoDB: The InnoDB memory heap is disabled 111012 14:20:53 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:53 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:53 InnoDB: Using Linux native AIO 111012 14:20:53 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:53 InnoDB: Completed initialization of buffer pool 111012 14:20:53 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:53 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:53 InnoDB: Waiting for the background threads to start 111012 14:20:54 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:54 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:54 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:54 [Note] Event Scheduler: Loaded 0 events 111012 14:20:54 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:54 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f5020f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:54 mysqld_safe Number of processes running now: 0 111012 14:20:54 mysqld_safe mysqld restarted 111012 14:20:54 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:54 [Note] Flashcache bypass: disabled 111012 14:20:54 [Note] Flashcache setup error is : ioctl failed 111012 14:20:54 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:54 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:54 InnoDB: The InnoDB memory heap is disabled 111012 14:20:54 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:54 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:54 InnoDB: Using Linux native AIO 111012 14:20:54 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:55 InnoDB: Completed initialization of buffer pool 111012 14:20:55 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:55 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:55 InnoDB: Waiting for the background threads to start 111012 14:20:56 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:56 [Note] Event Scheduler: Loaded 0 events 111012 14:20:56 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:56 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:56 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:56 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xde36eb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x50bfa0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:56 mysqld_safe Number of processes running now: 0 111012 14:20:56 mysqld_safe mysqld restarted 111012 14:20:56 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:56 [Note] Flashcache bypass: disabled 111012 14:20:56 [Note] Flashcache setup error is : ioctl failed 111012 14:20:56 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:56 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:56 InnoDB: The InnoDB memory heap is disabled 111012 14:20:56 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:56 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:56 InnoDB: Using Linux native AIO 111012 14:20:56 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:56 InnoDB: Completed initialization of buffer pool 111012 14:20:56 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:56 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:56 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:56 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:56 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:56 [Note] Event Scheduler: Loaded 0 events 111012 14:20:56 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:56 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1cd7a6c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x502190f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:56 mysqld_safe Number of processes running now: 0 111012 14:20:56 mysqld_safe mysqld restarted 111012 14:20:56 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:56 [Note] Flashcache bypass: disabled 111012 14:20:56 [Note] Flashcache setup error is : ioctl failed 111012 14:20:56 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:56 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:56 InnoDB: The InnoDB memory heap is disabled 111012 14:20:56 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:56 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:56 InnoDB: Using Linux native AIO 111012 14:20:57 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:57 InnoDB: Completed initialization of buffer pool 111012 14:20:57 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:57 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:57 InnoDB: Waiting for the background threads to start 111012 14:20:58 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:20:58 [Note] Event Scheduler: Loaded 0 events 111012 14:20:58 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:20:58 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:20:58 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:20:58 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1fe00d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419dc0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:20:58 mysqld_safe Number of processes running now: 0 111012 14:20:58 mysqld_safe mysqld restarted 111012 14:20:58 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:20:58 [Note] Flashcache bypass: disabled 111012 14:20:58 [Note] Flashcache setup error is : ioctl failed 111012 14:20:58 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:20:58 [Note] Plugin 'FEDERATED' is disabled. 111012 14:20:58 InnoDB: The InnoDB memory heap is disabled 111012 14:20:58 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:20:58 InnoDB: Compressed tables use zlib 1.2.3 111012 14:20:58 InnoDB: Using Linux native AIO 111012 14:20:58 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:20:58 InnoDB: Completed initialization of buffer pool 111012 14:20:58 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:20:58 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:20:59 InnoDB: Waiting for the background threads to start 111012 14:21:00 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:00 [Note] Event Scheduler: Loaded 0 events 111012 14:21:00 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:00 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:00 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:00 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x10a61d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40ed20f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:00 mysqld_safe Number of processes running now: 0 111012 14:21:00 mysqld_safe mysqld restarted 111012 14:21:00 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:00 [Note] Flashcache bypass: disabled 111012 14:21:00 [Note] Flashcache setup error is : ioctl failed 111012 14:21:00 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:00 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:00 InnoDB: The InnoDB memory heap is disabled 111012 14:21:00 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:00 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:00 InnoDB: Using Linux native AIO 111012 14:21:00 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:00 InnoDB: Completed initialization of buffer pool 111012 14:21:00 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:00 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:00 InnoDB: Waiting for the background threads to start 111012 14:21:01 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:01 [Note] Event Scheduler: Loaded 0 events 111012 14:21:01 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:01 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:01 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:01 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x22414c60 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fa160f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:01 mysqld_safe Number of processes running now: 0 111012 14:21:01 mysqld_safe mysqld restarted 111012 14:21:01 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:01 [Note] Flashcache bypass: disabled 111012 14:21:01 [Note] Flashcache setup error is : ioctl failed 111012 14:21:01 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:01 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:01 InnoDB: The InnoDB memory heap is disabled 111012 14:21:01 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:01 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:01 InnoDB: Using Linux native AIO 111012 14:21:01 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:02 InnoDB: Completed initialization of buffer pool 111012 14:21:02 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:02 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:02 InnoDB: Waiting for the background threads to start 111012 14:21:03 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:03 [Note] Event Scheduler: Loaded 0 events 111012 14:21:03 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:03 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:03 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:03 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x24805d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40dcd0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:03 mysqld_safe Number of processes running now: 0 111012 14:21:03 mysqld_safe mysqld restarted 111012 14:21:03 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:03 [Note] Flashcache bypass: disabled 111012 14:21:03 [Note] Flashcache setup error is : ioctl failed 111012 14:21:03 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:03 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:03 InnoDB: The InnoDB memory heap is disabled 111012 14:21:03 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:03 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:03 InnoDB: Using Linux native AIO 111012 14:21:03 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:03 InnoDB: Completed initialization of buffer pool 111012 14:21:03 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:03 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:03 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:03 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:03 [Note] Event Scheduler: Loaded 0 events 111012 14:21:03 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:03 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:03 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1b1cd620 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40f240f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:04 mysqld_safe Number of processes running now: 0 111012 14:21:04 mysqld_safe mysqld restarted 111012 14:21:04 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:04 [Note] Flashcache bypass: disabled 111012 14:21:04 [Note] Flashcache setup error is : ioctl failed 111012 14:21:04 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:04 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:04 InnoDB: The InnoDB memory heap is disabled 111012 14:21:04 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:04 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:04 InnoDB: Using Linux native AIO 111012 14:21:04 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:04 InnoDB: Completed initialization of buffer pool 111012 14:21:04 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:04 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:04 InnoDB: Waiting for the background threads to start 111012 14:21:05 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:05 [Note] Event Scheduler: Loaded 0 events 111012 14:21:05 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:05 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:05 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:05 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x23e8feb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40bae0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:05 mysqld_safe Number of processes running now: 0 111012 14:21:05 mysqld_safe mysqld restarted 111012 14:21:05 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:05 [Note] Flashcache bypass: disabled 111012 14:21:05 [Note] Flashcache setup error is : ioctl failed 111012 14:21:05 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:05 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:05 InnoDB: The InnoDB memory heap is disabled 111012 14:21:05 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:05 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:05 InnoDB: Using Linux native AIO 111012 14:21:05 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:05 InnoDB: Completed initialization of buffer pool 111012 14:21:05 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:05 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:06 InnoDB: Waiting for the background threads to start 111012 14:21:07 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:07 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:07 [Note] Event Scheduler: Loaded 0 events 111012 14:21:07 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:07 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:07 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x19aa9530 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5005d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:07 mysqld_safe Number of processes running now: 0 111012 14:21:07 mysqld_safe mysqld restarted 111012 14:21:07 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:07 [Note] Flashcache bypass: disabled 111012 14:21:07 [Note] Flashcache setup error is : ioctl failed 111012 14:21:07 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:07 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:07 InnoDB: The InnoDB memory heap is disabled 111012 14:21:07 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:07 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:07 InnoDB: Using Linux native AIO 111012 14:21:07 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:07 InnoDB: Completed initialization of buffer pool 111012 14:21:07 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:07 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:07 InnoDB: Waiting for the background threads to start 111012 14:21:08 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:08 [Note] Event Scheduler: Loaded 0 events 111012 14:21:08 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:08 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:08 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:08 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x19e87eb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419400f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:08 mysqld_safe Number of processes running now: 0 111012 14:21:08 mysqld_safe mysqld restarted 111012 14:21:08 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:08 [Note] Flashcache bypass: disabled 111012 14:21:08 [Note] Flashcache setup error is : ioctl failed 111012 14:21:08 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:08 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:08 InnoDB: The InnoDB memory heap is disabled 111012 14:21:08 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:08 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:08 InnoDB: Using Linux native AIO 111012 14:21:08 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:09 InnoDB: Completed initialization of buffer pool 111012 14:21:09 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:09 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:09 InnoDB: Waiting for the background threads to start 111012 14:21:10 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:10 [Note] Event Scheduler: Loaded 0 events 111012 14:21:10 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:10 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:10 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:10 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x22015d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x500260f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:10 mysqld_safe Number of processes running now: 0 111012 14:21:10 mysqld_safe mysqld restarted 111012 14:21:10 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:10 [Note] Flashcache bypass: disabled 111012 14:21:10 [Note] Flashcache setup error is : ioctl failed 111012 14:21:10 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:10 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:10 InnoDB: The InnoDB memory heap is disabled 111012 14:21:10 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:10 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:10 InnoDB: Using Linux native AIO 111012 14:21:10 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:10 InnoDB: Completed initialization of buffer pool 111012 14:21:10 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:10 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:10 InnoDB: Waiting for the background threads to start 111012 14:21:11 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:11 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:11 [Note] Event Scheduler: Loaded 0 events 111012 14:21:11 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:11 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:11 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x500540f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:12 mysqld_safe Number of processes running now: 0 111012 14:21:12 mysqld_safe mysqld restarted 111012 14:21:12 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:12 [Note] Flashcache bypass: disabled 111012 14:21:12 [Note] Flashcache setup error is : ioctl failed 111012 14:21:12 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:12 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:12 InnoDB: The InnoDB memory heap is disabled 111012 14:21:12 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:12 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:12 InnoDB: Using Linux native AIO 111012 14:21:12 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:12 InnoDB: Completed initialization of buffer pool 111012 14:21:12 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:12 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:12 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:12 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:12 [Note] Event Scheduler: Loaded 0 events 111012 14:21:12 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:12 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:12 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x189c2cf0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x41bc30f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:12 mysqld_safe Number of processes running now: 0 111012 14:21:12 mysqld_safe mysqld restarted 111012 14:21:12 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:12 [Note] Flashcache bypass: disabled 111012 14:21:12 [Note] Flashcache setup error is : ioctl failed 111012 14:21:12 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:12 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:12 InnoDB: The InnoDB memory heap is disabled 111012 14:21:12 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:12 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:12 InnoDB: Using Linux native AIO 111012 14:21:12 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:12 InnoDB: Completed initialization of buffer pool 111012 14:21:12 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:12 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:13 InnoDB: Waiting for the background threads to start 111012 14:21:14 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:14 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:14 [Note] Event Scheduler: Loaded 0 events 111012 14:21:14 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:14 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:14 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x417850f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:14 mysqld_safe Number of processes running now: 0 111012 14:21:14 mysqld_safe mysqld restarted 111012 14:21:14 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:14 [Note] Flashcache bypass: disabled 111012 14:21:14 [Note] Flashcache setup error is : ioctl failed 111012 14:21:14 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:14 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:14 InnoDB: The InnoDB memory heap is disabled 111012 14:21:14 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:14 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:14 InnoDB: Using Linux native AIO 111012 14:21:14 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:14 InnoDB: Completed initialization of buffer pool 111012 14:21:14 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:14 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:14 InnoDB: Waiting for the background threads to start 111012 14:21:15 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:15 [Note] Event Scheduler: Loaded 0 events 111012 14:21:15 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:15 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:15 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:15 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xc44ad90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x501d00f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:15 mysqld_safe Number of processes running now: 0 111012 14:21:15 mysqld_safe mysqld restarted 111012 14:21:15 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:15 [Note] Flashcache bypass: disabled 111012 14:21:15 [Note] Flashcache setup error is : ioctl failed 111012 14:21:15 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:15 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:15 InnoDB: The InnoDB memory heap is disabled 111012 14:21:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:15 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:15 InnoDB: Using Linux native AIO 111012 14:21:15 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:16 InnoDB: Completed initialization of buffer pool 111012 14:21:16 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:16 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:16 InnoDB: Waiting for the background threads to start 111012 14:21:17 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:17 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:17 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:17 [Note] Event Scheduler: Loaded 0 events 111012 14:21:17 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:17 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x402860f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:17 mysqld_safe Number of processes running now: 0 111012 14:21:17 mysqld_safe mysqld restarted 111012 14:21:17 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:17 [Note] Flashcache bypass: disabled 111012 14:21:17 [Note] Flashcache setup error is : ioctl failed 111012 14:21:17 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:17 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:17 InnoDB: The InnoDB memory heap is disabled 111012 14:21:17 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:17 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:17 InnoDB: Using Linux native AIO 111012 14:21:17 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:17 InnoDB: Completed initialization of buffer pool 111012 14:21:17 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:17 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:17 InnoDB: Waiting for the background threads to start 111012 14:21:18 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:18 [Note] Event Scheduler: Loaded 0 events 111012 14:21:18 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:18 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:18 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:18 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xb2a5eb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x506360f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:19 mysqld_safe Number of processes running now: 0 111012 14:21:19 mysqld_safe mysqld restarted 111012 14:21:19 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:19 [Note] Flashcache bypass: disabled 111012 14:21:19 [Note] Flashcache setup error is : ioctl failed 111012 14:21:19 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:19 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:19 InnoDB: The InnoDB memory heap is disabled 111012 14:21:19 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:19 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:19 InnoDB: Using Linux native AIO 111012 14:21:19 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:19 InnoDB: Completed initialization of buffer pool 111012 14:21:19 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:19 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:19 InnoDB: Waiting for the background threads to start 111012 14:21:20 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:20 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:20 [Note] Event Scheduler: Loaded 0 events 111012 14:21:20 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:20 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:20 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1c9cadb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x500a20f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:20 mysqld_safe Number of processes running now: 0 111012 14:21:20 mysqld_safe mysqld restarted 111012 14:21:20 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:20 [Note] Flashcache bypass: disabled 111012 14:21:20 [Note] Flashcache setup error is : ioctl failed 111012 14:21:20 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:20 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:20 InnoDB: The InnoDB memory heap is disabled 111012 14:21:20 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:20 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:20 InnoDB: Using Linux native AIO 111012 14:21:20 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:20 InnoDB: Completed initialization of buffer pool 111012 14:21:20 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:20 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:21 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:21 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:21 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:21 [Note] Event Scheduler: Loaded 0 events 111012 14:21:21 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:21 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab400008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fda70f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:21 mysqld_safe Number of processes running now: 0 111012 14:21:21 mysqld_safe mysqld restarted 111012 14:21:21 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:21 [Note] Flashcache bypass: disabled 111012 14:21:21 [Note] Flashcache setup error is : ioctl failed 111012 14:21:21 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:21 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:21 InnoDB: The InnoDB memory heap is disabled 111012 14:21:21 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:21 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:21 InnoDB: Using Linux native AIO 111012 14:21:21 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:21 InnoDB: Completed initialization of buffer pool 111012 14:21:21 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:21 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:21 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:21 [Note] Event Scheduler: Loaded 0 events 111012 14:21:21 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:21 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:21 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:21 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x75a6b20 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5029e0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:21 mysqld_safe Number of processes running now: 0 111012 14:21:21 mysqld_safe mysqld restarted 111012 14:21:21 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:21 [Note] Flashcache bypass: disabled 111012 14:21:21 [Note] Flashcache setup error is : ioctl failed 111012 14:21:21 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:21 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:21 InnoDB: The InnoDB memory heap is disabled 111012 14:21:21 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:21 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:21 InnoDB: Using Linux native AIO 111012 14:21:21 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:22 InnoDB: Completed initialization of buffer pool 111012 14:21:22 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:22 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:22 InnoDB: Waiting for the background threads to start 111012 14:21:23 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:23 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:23 [Note] Event Scheduler: Loaded 0 events 111012 14:21:23 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:23 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:23 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1a4eebf0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x508960f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:23 mysqld_safe Number of processes running now: 0 111012 14:21:23 mysqld_safe mysqld restarted 111012 14:21:23 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:23 [Note] Flashcache bypass: disabled 111012 14:21:23 [Note] Flashcache setup error is : ioctl failed 111012 14:21:23 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:23 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:23 InnoDB: The InnoDB memory heap is disabled 111012 14:21:23 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:23 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:23 InnoDB: Using Linux native AIO 111012 14:21:23 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:23 InnoDB: Completed initialization of buffer pool 111012 14:21:23 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:23 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:23 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:23 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:23 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:23 [Note] Event Scheduler: Loaded 0 events 111012 14:21:23 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:23 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab400008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x507840f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:24 mysqld_safe Number of processes running now: 0 111012 14:21:24 mysqld_safe mysqld restarted 111012 14:21:24 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:24 [Note] Flashcache bypass: disabled 111012 14:21:24 [Note] Flashcache setup error is : ioctl failed 111012 14:21:24 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:24 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:24 InnoDB: The InnoDB memory heap is disabled 111012 14:21:24 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:24 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:24 InnoDB: Using Linux native AIO 111012 14:21:24 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:24 InnoDB: Completed initialization of buffer pool 111012 14:21:24 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:24 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:24 InnoDB: Waiting for the background threads to start 111012 14:21:25 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:25 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:25 [Note] Event Scheduler: Loaded 0 events 111012 14:21:25 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:25 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:25 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x69d1310 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4128f0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:25 mysqld_safe Number of processes running now: 0 111012 14:21:25 mysqld_safe mysqld restarted 111012 14:21:25 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:25 [Note] Flashcache bypass: disabled 111012 14:21:25 [Note] Flashcache setup error is : ioctl failed 111012 14:21:25 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:25 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:25 InnoDB: The InnoDB memory heap is disabled 111012 14:21:25 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:25 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:25 InnoDB: Using Linux native AIO 111012 14:21:25 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:25 InnoDB: Completed initialization of buffer pool 111012 14:21:25 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:25 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:26 InnoDB: Waiting for the background threads to start 111012 14:21:27 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:27 [Note] Event Scheduler: Loaded 0 events 111012 14:21:27 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:27 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:27 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:27 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1ea95eb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fce70f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:27 mysqld_safe Number of processes running now: 0 111012 14:21:27 mysqld_safe mysqld restarted 111012 14:21:27 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:27 [Note] Flashcache bypass: disabled 111012 14:21:27 [Note] Flashcache setup error is : ioctl failed 111012 14:21:27 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:27 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:27 InnoDB: The InnoDB memory heap is disabled 111012 14:21:27 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:27 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:27 InnoDB: Using Linux native AIO 111012 14:21:27 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:27 InnoDB: Completed initialization of buffer pool 111012 14:21:27 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:27 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:27 InnoDB: Waiting for the background threads to start 111012 14:21:28 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:28 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:28 [Note] Event Scheduler: Loaded 0 events 111012 14:21:28 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:28 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:28 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4101d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:28 mysqld_safe Number of processes running now: 0 111012 14:21:28 mysqld_safe mysqld restarted 111012 14:21:28 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:28 [Note] Flashcache bypass: disabled 111012 14:21:28 [Note] Flashcache setup error is : ioctl failed 111012 14:21:28 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:28 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:28 InnoDB: The InnoDB memory heap is disabled 111012 14:21:28 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:28 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:28 InnoDB: Using Linux native AIO 111012 14:21:28 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:29 InnoDB: Completed initialization of buffer pool 111012 14:21:29 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:29 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:29 InnoDB: Waiting for the background threads to start 111012 14:21:30 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:30 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:30 [Note] Event Scheduler: Loaded 0 events 111012 14:21:30 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:30 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:30 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f12f0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:30 mysqld_safe Number of processes running now: 0 111012 14:21:30 mysqld_safe mysqld restarted 111012 14:21:30 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:30 [Note] Flashcache bypass: disabled 111012 14:21:30 [Note] Flashcache setup error is : ioctl failed 111012 14:21:30 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:30 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:30 InnoDB: The InnoDB memory heap is disabled 111012 14:21:30 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:30 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:30 InnoDB: Using Linux native AIO 111012 14:21:30 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:30 InnoDB: Completed initialization of buffer pool 111012 14:21:30 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:30 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:30 InnoDB: Waiting for the background threads to start 111012 14:21:31 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:32 [Note] Event Scheduler: Loaded 0 events 111012 14:21:32 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:32 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:32 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:32 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x21a0feb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x416f50f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:32 mysqld_safe Number of processes running now: 0 111012 14:21:32 mysqld_safe mysqld restarted 111012 14:21:32 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:32 [Note] Flashcache bypass: disabled 111012 14:21:32 [Note] Flashcache setup error is : ioctl failed 111012 14:21:32 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:32 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:32 InnoDB: The InnoDB memory heap is disabled 111012 14:21:32 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:32 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:32 InnoDB: Using Linux native AIO 111012 14:21:32 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:32 InnoDB: Completed initialization of buffer pool 111012 14:21:32 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:32 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:32 InnoDB: Waiting for the background threads to start 111012 14:21:33 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:33 [Note] Event Scheduler: Loaded 0 events 111012 14:21:33 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:33 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:33 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:33 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x18c2dd90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5014b0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:33 mysqld_safe Number of processes running now: 0 111012 14:21:33 mysqld_safe mysqld restarted 111012 14:21:33 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:33 [Note] Flashcache bypass: disabled 111012 14:21:33 [Note] Flashcache setup error is : ioctl failed 111012 14:21:33 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:33 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:33 InnoDB: The InnoDB memory heap is disabled 111012 14:21:33 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:33 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:33 InnoDB: Using Linux native AIO 111012 14:21:33 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:33 InnoDB: Completed initialization of buffer pool 111012 14:21:33 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:34 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:34 InnoDB: Waiting for the background threads to start 111012 14:21:35 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:35 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:35 [Note] Event Scheduler: Loaded 0 events 111012 14:21:35 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:35 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:35 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1d6f8120 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x505390f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:35 mysqld_safe Number of processes running now: 0 111012 14:21:35 mysqld_safe mysqld restarted 111012 14:21:35 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:35 [Note] Flashcache bypass: disabled 111012 14:21:35 [Note] Flashcache setup error is : ioctl failed 111012 14:21:35 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:35 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:35 InnoDB: The InnoDB memory heap is disabled 111012 14:21:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:35 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:35 InnoDB: Using Linux native AIO 111012 14:21:35 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:35 InnoDB: Completed initialization of buffer pool 111012 14:21:35 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:35 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:35 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:35 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:35 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:35 [Note] Event Scheduler: Loaded 0 events 111012 14:21:35 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:35 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xfcacd70 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x503bb0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:35 mysqld_safe Number of processes running now: 0 111012 14:21:35 mysqld_safe mysqld restarted 111012 14:21:35 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:35 [Note] Flashcache bypass: disabled 111012 14:21:35 [Note] Flashcache setup error is : ioctl failed 111012 14:21:35 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:35 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:35 InnoDB: The InnoDB memory heap is disabled 111012 14:21:35 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:35 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:35 InnoDB: Using Linux native AIO 111012 14:21:35 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:36 InnoDB: Completed initialization of buffer pool 111012 14:21:36 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:36 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:36 InnoDB: Waiting for the background threads to start 111012 14:21:37 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:37 [Note] Event Scheduler: Loaded 0 events 111012 14:21:37 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:37 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:37 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:37 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1b1afeb0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x5033e0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:37 mysqld_safe Number of processes running now: 0 111012 14:21:37 mysqld_safe mysqld restarted 111012 14:21:37 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:37 [Note] Flashcache bypass: disabled 111012 14:21:37 [Note] Flashcache setup error is : ioctl failed 111012 14:21:37 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:37 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:37 InnoDB: The InnoDB memory heap is disabled 111012 14:21:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:37 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:37 InnoDB: Using Linux native AIO 111012 14:21:37 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:37 InnoDB: Completed initialization of buffer pool 111012 14:21:37 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:37 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:38 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:38 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:38 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:38 [Note] Event Scheduler: Loaded 0 events 111012 14:21:38 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:38 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x18aed810 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x408260f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:38 mysqld_safe Number of processes running now: 0 111012 14:21:38 mysqld_safe mysqld restarted 111012 14:21:38 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:38 [Note] Flashcache bypass: disabled 111012 14:21:38 [Note] Flashcache setup error is : ioctl failed 111012 14:21:38 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:38 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:38 InnoDB: The InnoDB memory heap is disabled 111012 14:21:38 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:38 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:38 InnoDB: Using Linux native AIO 111012 14:21:38 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:38 InnoDB: Completed initialization of buffer pool 111012 14:21:38 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:38 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:38 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:38 [Note] Event Scheduler: Loaded 0 events 111012 14:21:38 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:38 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:38 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:38 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xfca7af0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x50b0d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:38 mysqld_safe Number of processes running now: 0 111012 14:21:38 mysqld_safe mysqld restarted 111012 14:21:38 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:38 [Note] Flashcache bypass: disabled 111012 14:21:38 [Note] Flashcache setup error is : ioctl failed 111012 14:21:38 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:38 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:38 InnoDB: The InnoDB memory heap is disabled 111012 14:21:38 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:38 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:38 InnoDB: Using Linux native AIO 111012 14:21:38 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:39 InnoDB: Completed initialization of buffer pool 111012 14:21:39 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:39 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:39 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:39 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:39 [Note] Event Scheduler: Loaded 0 events 111012 14:21:39 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:39 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:39 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1629a0a0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f9920f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:39 mysqld_safe Number of processes running now: 0 111012 14:21:39 mysqld_safe mysqld restarted 111012 14:21:39 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:39 [Note] Flashcache bypass: disabled 111012 14:21:39 [Note] Flashcache setup error is : ioctl failed 111012 14:21:39 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:39 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:39 InnoDB: The InnoDB memory heap is disabled 111012 14:21:39 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:39 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:39 InnoDB: Using Linux native AIO 111012 14:21:39 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:39 InnoDB: Completed initialization of buffer pool 111012 14:21:39 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:39 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:39 InnoDB: Waiting for the background threads to start 111012 14:21:40 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:40 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:40 [Note] Event Scheduler: Loaded 0 events 111012 14:21:40 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:40 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:40 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xba6d7b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f94a0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:40 mysqld_safe Number of processes running now: 0 111012 14:21:40 mysqld_safe mysqld restarted 111012 14:21:40 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:40 [Note] Flashcache bypass: disabled 111012 14:21:40 [Note] Flashcache setup error is : ioctl failed 111012 14:21:40 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:40 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:40 InnoDB: The InnoDB memory heap is disabled 111012 14:21:40 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:40 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:40 InnoDB: Using Linux native AIO 111012 14:21:41 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:41 InnoDB: Completed initialization of buffer pool 111012 14:21:41 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:41 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:41 InnoDB: Waiting for the background threads to start 111012 14:21:42 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:42 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:42 [Note] Event Scheduler: Loaded 0 events 111012 14:21:42 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:42 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:42 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xe5a9b00 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419760f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:42 mysqld_safe Number of processes running now: 0 111012 14:21:42 mysqld_safe mysqld restarted 111012 14:21:42 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:42 [Note] Flashcache bypass: disabled 111012 14:21:42 [Note] Flashcache setup error is : ioctl failed 111012 14:21:42 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:42 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:42 InnoDB: The InnoDB memory heap is disabled 111012 14:21:42 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:42 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:42 InnoDB: Using Linux native AIO 111012 14:21:42 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:42 InnoDB: Completed initialization of buffer pool 111012 14:21:42 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:42 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:43 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:43 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:43 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:43 [Note] Event Scheduler: Loaded 0 events 111012 14:21:43 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:43 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0010b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x419330f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:43 mysqld_safe Number of processes running now: 0 111012 14:21:43 mysqld_safe mysqld restarted 111012 14:21:43 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:43 [Note] Flashcache bypass: disabled 111012 14:21:43 [Note] Flashcache setup error is : ioctl failed 111012 14:21:43 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:43 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:43 InnoDB: The InnoDB memory heap is disabled 111012 14:21:43 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:43 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:43 InnoDB: Using Linux native AIO 111012 14:21:43 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:43 InnoDB: Completed initialization of buffer pool 111012 14:21:43 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:43 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:43 InnoDB: Waiting for the background threads to start 111012 14:21:44 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:44 [Note] Event Scheduler: Loaded 0 events 111012 14:21:44 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:44 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:44 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:44 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x238607e0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x40c5d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:44 mysqld_safe Number of processes running now: 0 111012 14:21:44 mysqld_safe mysqld restarted 111012 14:21:44 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:44 [Note] Flashcache bypass: disabled 111012 14:21:44 [Note] Flashcache setup error is : ioctl failed 111012 14:21:44 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:44 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:44 InnoDB: The InnoDB memory heap is disabled 111012 14:21:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:44 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:44 InnoDB: Using Linux native AIO 111012 14:21:44 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:45 InnoDB: Completed initialization of buffer pool 111012 14:21:45 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:45 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:45 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:45 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:45 [Note] Event Scheduler: Loaded 0 events 111012 14:21:45 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:45 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:45 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xb8db980 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x415090f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:45 mysqld_safe Number of processes running now: 0 111012 14:21:45 mysqld_safe mysqld restarted 111012 14:21:45 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:45 [Note] Flashcache bypass: disabled 111012 14:21:45 [Note] Flashcache setup error is : ioctl failed 111012 14:21:45 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:45 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:45 InnoDB: The InnoDB memory heap is disabled 111012 14:21:45 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:45 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:45 InnoDB: Using Linux native AIO 111012 14:21:45 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:45 InnoDB: Completed initialization of buffer pool 111012 14:21:45 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:45 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:45 InnoDB: Waiting for the background threads to start 111012 14:21:46 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:46 [Note] Event Scheduler: Loaded 0 events 111012 14:21:46 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:46 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:46 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:46 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xea28d80 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f7b70f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:46 mysqld_safe Number of processes running now: 0 111012 14:21:46 mysqld_safe mysqld restarted 111012 14:21:46 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:46 [Note] Flashcache bypass: disabled 111012 14:21:46 [Note] Flashcache setup error is : ioctl failed 111012 14:21:46 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:47 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:47 InnoDB: The InnoDB memory heap is disabled 111012 14:21:47 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:47 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:47 InnoDB: Using Linux native AIO 111012 14:21:47 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:47 InnoDB: Completed initialization of buffer pool 111012 14:21:47 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:47 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:47 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:47 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:47 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:47 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x145cad00 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x417700f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] 111012 14:21:47 [Note] Event Scheduler: Loaded 0 events 111012 14:21:47 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:47 mysqld_safe Number of processes running now: 0 111012 14:21:47 mysqld_safe mysqld restarted 111012 14:21:47 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:47 [Note] Flashcache bypass: disabled 111012 14:21:47 [Note] Flashcache setup error is : ioctl failed 111012 14:21:47 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:47 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:47 InnoDB: The InnoDB memory heap is disabled 111012 14:21:47 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:47 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:47 InnoDB: Using Linux native AIO 111012 14:21:47 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:47 InnoDB: Completed initialization of buffer pool 111012 14:21:47 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:47 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:48 InnoDB: Waiting for the background threads to start 111012 14:21:49 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:49 [Note] Event Scheduler: Loaded 0 events 111012 14:21:49 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:49 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:49 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:49 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xb4c28a0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x418e00f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:49 mysqld_safe Number of processes running now: 0 111012 14:21:49 mysqld_safe mysqld restarted 111012 14:21:49 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:49 [Note] Flashcache bypass: disabled 111012 14:21:49 [Note] Flashcache setup error is : ioctl failed 111012 14:21:49 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:49 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:49 InnoDB: The InnoDB memory heap is disabled 111012 14:21:49 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:49 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:49 InnoDB: Using Linux native AIO 111012 14:21:49 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:49 InnoDB: Completed initialization of buffer pool 111012 14:21:49 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:49 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:49 InnoDB: Waiting for the background threads to start 111012 14:21:50 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:50 [Note] Event Scheduler: Loaded 0 events 111012 14:21:50 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:50 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:50 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:50 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x98ba1b0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4fe910f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:50 mysqld_safe Number of processes running now: 0 111012 14:21:50 mysqld_safe mysqld restarted 111012 14:21:50 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:50 [Note] Flashcache bypass: disabled 111012 14:21:50 [Note] Flashcache setup error is : ioctl failed 111012 14:21:50 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:50 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:50 InnoDB: The InnoDB memory heap is disabled 111012 14:21:50 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:50 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:50 InnoDB: Using Linux native AIO 111012 14:21:50 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:51 InnoDB: Completed initialization of buffer pool 111012 14:21:51 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:51 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:51 InnoDB: Waiting for the background threads to start 111012 14:21:52 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:52 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:52 [Note] Event Scheduler: Loaded 0 events 111012 14:21:52 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:52 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:52 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab3c0008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x4f88d0f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:52 mysqld_safe Number of processes running now: 0 111012 14:21:52 mysqld_safe mysqld restarted 111012 14:21:52 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:52 [Note] Flashcache bypass: disabled 111012 14:21:52 [Note] Flashcache setup error is : ioctl failed 111012 14:21:52 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:52 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:52 InnoDB: The InnoDB memory heap is disabled 111012 14:21:52 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:52 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:52 InnoDB: Using Linux native AIO 111012 14:21:52 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:52 InnoDB: Completed initialization of buffer pool 111012 14:21:52 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:52 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:52 InnoDB: Waiting for the background threads to start 111012 14:21:53 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:53 [Note] Event Scheduler: Loaded 0 events 111012 14:21:53 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:53 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:53 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:53 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x1cb65d90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x410870f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:53 mysqld_safe Number of processes running now: 0 111012 14:21:53 mysqld_safe mysqld restarted 111012 14:21:53 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:54 [Note] Flashcache bypass: disabled 111012 14:21:54 [Note] Flashcache setup error is : ioctl failed 111012 14:21:54 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:54 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:54 InnoDB: The InnoDB memory heap is disabled 111012 14:21:54 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:54 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:54 InnoDB: Using Linux native AIO 111012 14:21:54 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:54 InnoDB: Completed initialization of buffer pool 111012 14:21:54 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:54 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:54 InnoDB: Waiting for the background threads to start 111012 14:21:55 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:55 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:55 [Note] Event Scheduler: Loaded 0 events 111012 14:21:55 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:55 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:55 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xd0ce790 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x506610f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:55 mysqld_safe Number of processes running now: 0 111012 14:21:55 mysqld_safe mysqld restarted 111012 14:21:55 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:55 [Note] Flashcache bypass: disabled 111012 14:21:55 [Note] Flashcache setup error is : ioctl failed 111012 14:21:55 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:55 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:55 InnoDB: The InnoDB memory heap is disabled 111012 14:21:55 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:55 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:55 InnoDB: Using Linux native AIO 111012 14:21:55 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:55 InnoDB: Completed initialization of buffer pool 111012 14:21:55 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:55 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:56 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:56 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:56 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:56 [Note] Event Scheduler: Loaded 0 events 111012 14:21:56 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:56 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2aab40000990 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x501540f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 111012 14:21:56 mysqld_safe Number of processes running now: 0 111012 14:21:56 mysqld_safe mysqld restarted 111012 14:21:56 [Warning] option 'max_relay_log_size': unsigned value 2147483648 adjusted to 1073741824 111012 14:21:56 [Note] Flashcache bypass: disabled 111012 14:21:56 [Note] Flashcache setup error is : ioctl failed 111012 14:21:56 [Warning] You need to use --log-bin to make --binlog-format work. 111012 14:21:56 [Note] Plugin 'FEDERATED' is disabled. 111012 14:21:56 InnoDB: The InnoDB memory heap is disabled 111012 14:21:56 InnoDB: Mutexes and rw_locks use GCC atomic builtins 111012 14:21:56 InnoDB: Compressed tables use zlib 1.2.3 111012 14:21:56 InnoDB: Using Linux native AIO 111012 14:21:56 InnoDB: Initializing buffer pool, size = 2.0G 111012 14:21:56 InnoDB: Completed initialization of buffer pool 111012 14:21:56 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 111012 14:21:56 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 534536256, file name mysql-bin.000068 InnoDB: and relay log file InnoDB: position 0 534536402, file name /var/log/mysql/binary/mysql-relay-dbslave1.000089 111012 14:21:56 Percona XtraDB (http://www.percona.com) 1.1.8-20.1 started; log sequence number 108233728750 111012 14:21:56 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000068' at position 534536256, relay log '/var/log/mysql/binary/mysql-relay-dbslave1.000089' position: 534536402 111012 14:21:56 [Note] Event Scheduler: Loaded 0 events 111012 14:21:56 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.15-55' socket: '/var/db/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release rel21.0, Revision 158 111012 14:21:56 [Note] Slave I/O thread: connected to master 'repl@10.225.206.7:3306',replication started in log 'mysql-bin.000073' at position 1074397168 111012 14:21:56 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=100 thread_count=0 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1057963 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x8461380 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x410270f8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7cdb99] /usr/sbin/mysqld(handle_segfault+0x379)[0x500b59] /lib64/libpthread.so.0[0x3a3260eb10] /usr/sbin/mysqld(_ZNK9table_def15compatible_withEP3THDP14Relay_log_infoP5TABLEPS5_+0x31d)[0x7b143d] /usr/sbin/mysqld(_ZN14Rows_log_event14do_apply_eventEPK14Relay_log_info+0xf9c)[0x74a0fc] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x152)[0x516be2] /usr/sbin/mysqld[0x5192f2] /usr/sbin/mysqld(handle_slave_sql+0xc19)[0x5208f9] /lib64/libpthread.so.0[0x3a3260673d] /lib64/libc.so.6(clone+0x6d)[0x3a312d44bd] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query ((nil)): is an invalid pointer Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.