120316 2:34:12 [Warning] Aborted connection 25763 to db: 'mysql' user: 'config_root' host: 'localhost' (Got an error reading communication packets) *** glibc detected *** /usr/sbin/mysqld: malloc(): memory corruption: 0x00007f63284952c0 *** ======= Backtrace: ========= /lib/libc.so.6(+0x71bd6)[0x7f6365dd6bd6] /lib/libc.so.6(+0x74c6d)[0x7f6365dd9c6d] /lib/libc.so.6(__libc_malloc+0x70)[0x7f6365ddba30] /usr/sbin/mysqld(my_malloc+0x32)[0x8043f2] /usr/sbin/mysqld(_mi_write_blob_record+0x39)[0x7c05b9] /usr/sbin/mysqld(mi_write+0x828)[0x7da4a8] /usr/sbin/mysqld(_ZN9ha_myisam9write_rowEPh+0x43)[0x7a8183] /usr/sbin/mysqld(_ZN7handler12ha_write_rowEPh+0x5d)[0x694a0d] /usr/sbin/mysqld(_Z12write_recordP3THDP5TABLEP12st_copy_info+0x62)[0x57f512] /usr/sbin/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0xaff)[0x58640f] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x89f)[0x5951ef] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x33c)[0x59935c] /usr/sbin/mysqld(_ZN15Query_log_event14do_apply_eventEPK14Relay_log_infoPKcj+0xd5c)[0x73e4ac] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x125)[0x533555] /usr/sbin/mysqld[0x537ed1] /usr/sbin/mysqld(handle_slave_sql+0xa7b)[0x53a17b] /lib/libpthread.so.0(+0x68ca)[0x7f6366b938ca] /lib/libc.so.6(clone+0x6d)[0x7f6365e3492d] ======= Memory map: ======== 00400000-00d3f000 r-xp 00000000 fe:05 36176594 /usr/sbin/mysqld 00f3e000-0105a000 rw-p 0093e000 fe:05 36176594 /usr/sbin/mysqld 0105a000-01088000 rw-p 00000000 00:00 0 016e3000-06dd5000 rw-p 00000000 00:00 0 [heap] 7f6328000000-7f6328d4c000 rw-p 00000000 00:00 0 7f6328d4c000-7f632c000000 ---p 00000000 00:00 0 7f6330000000-7f6330bde000 rw-p 00000000 00:00 0 7f6330bde000-7f6334000000 ---p 00000000 00:00 0 7f6335cdd000-7f6335cde000 ---p 00000000 00:00 0 7f6335cde000-7f63364de000 rw-p 00000000 00:00 0 7f63364de000-7f63364df000 ---p 00000000 00:00 0 7f63364df000-7f6336cdf000 rw-p 00000000 00:00 0 7f6336cdf000-7f6336ce0000 ---p 00000000 00:00 0 7f6336ce0000-7f63374e0000 rw-p 00000000 00:00 0 7f63374e0000-7f63374e1000 ---p 00000000 00:00 0 7f63374e1000-7f6337ce1000 rw-p 00000000 00:00 0 7f6337ce1000-7f6337ce2000 ---p 00000000 00:00 0 7f6337ce2000-7f633891e000 rw-p 00000000 00:00 0 7f6338bef000-7f6338bf0000 ---p 00000000 00:00 0 7f6338bf0000-7f63393f0000 rw-p 00000000 00:00 0 7f63393f0000-7f63393f1000 ---p 00000000 00:00 0 7f63393f1000-7f6339bf1000 rw-p 00000000 00:00 0 7f6339bf1000-7f6339bf2000 ---p 00000000 00:00 0 7f6339bf2000-7f633a3f2000 rw-p 00000000 00:00 0 7f633a3f2000-7f633a3f3000 ---p 00000000 00:00 0 7f633a3f3000-7f633abf3000 rw-p 00000000 00:00 0 7f633abf3000-7f633abf4000 ---p 00000000 00:00 0 7f633abf4000-7f633b3f4000 rw-p 00000000 00:00 0 7f633b3f4000-7f633b3f5000 ---p 00000000 00:00 0 7f633b3f5000-7f633bbf5000 rw-p 00000000 00:00 0 7f633bbf5000-7f633bbf6000 ---p 00000000 00:00 0 7f633bbf6000-7f633c3f6000 rw-p 00000000 00:00 0 7f633c3f6000-7f633c3f7000 ---p 00000000 00:00 0 7f633c3f7000-7f633cbf7000 rw-p 00000000 00:00 0 7f633cbf7000-7f633cbf8000 ---p 00000000 00:00 0 7f633cbf8000-7f633d3f8000 rw-p 00000000 00:00 0 7f633d3f8000-7f633d3f9000 ---p 00000000 00:00 0 7f633d3f9000-7f6364b16000 rw-p 00000000 00:00 0 7f6364b16000-7f6364b22000 r-xp 00000000 fe:05 36176624 /lib/libnss_files-2.11.3.so 7f6364b22000-7f6364d21000 ---p 0000c000 fe:05 36176624 /lib/libnss_files-2.11.3.so 7f6364d21000-7f6364d22000 r--p 0000b000 fe:05 36176624 /lib/libnss_files-2.11.3.so 7f6364d22000-7f6364d23000 rw-p 0000c000 fe:05 36176624 /lib/libnss_files-2.11.3.so 7f6364d23000-7f6364d2d000 r-xp 00000000 fe:05 36177657 /lib/libnss_nis-2.11.3.so 7f6364d2d000-7f6364f2c000 ---p 0000a000 fe:05 36177657 /lib/libnss_nis-2.11.3.so 7f6364f2c000-7f6364f2d000 r--p 00009000 fe:05 36177657 /lib/libnss_nis-2.11.3.so 7f6364f2d000-7f6364f2e000 rw-p 0000a000 fe:05 36177657 /lib/libnss_nis-2.11.3.so 7f6364f2e000-7f6364f43000 r-xp 00000000 fe:05 36177647 /lib/libnsl-2.11.3.so 7f6364f43000-7f6365142000 ---p 00015000 fe:05 36177647 /lib/libnsl-2.11.3.so 7f6365142000-7f6365143000 r--p 00014000 fe:05 36177647 /lib/libnsl-2.11.3.so 7f6365143000-7f6365144000 rw-p 00015000 fe:05 36177647 /lib/libnsl-2.11.3.so 7f6365144000-7f6365146000 rw-p 00000000 00:00 0 7f6365146000-7f636514d000 r-xp 00000000 fe:05 36177641 /lib/libnss_compat-2.11.3.so 7f636514d000-7f636534c000 ---p 00007000 fe:05 36177641 /lib/libnss_compat-2.11.3.so 7f636534c000-7f636534d000 r--p 00006000 fe:05 36177641 /lib/libnss_compat-2.11.3.so 7f636534d000-7f636534e000 rw-p 00007000 fe:05 36177641 /lib/libnss_compat-2.11.3.so 7f636534e000-7f6365364000 r-xp 00000000 fe:05 36176385 /lib/libgcc_s.so.1 7f6365364000-7f6365563000 ---p 00016000 fe:05 36176385 /lib/libgcc_s.so.1 7f6365563000-7f6365564000 rw-p 00015000 fe:05 36176385 /lib/libgcc_s.so.1 7f6365564000-7f6365565000 ---p 00000000 00:00 0 7f6365565000-7f6365d65000 rw-p 00000000 00:00 0 7f6365d65000-7f6365ebe000 r-xp 00000000 fe:05 36177642 /lib/libc-2.11.3.so 7f6365ebe000-7f63660bd000 ---p 00159000 fe:05 36177642 /lib/libc-2.11.3.so 7f63660bd000-7f63660c1000 r--p 00158000 fe:05 36177642 /lib/libc-2.11.3.so 7f63660c1000-7f63660c2000 rw-p 0015c000 fe:05 36177642 /lib/libc-2.11.3.so 7f63660c2000-7f63660c7000 rw-p 00000000 00:00 0 7f63660c7000-7f63660c8000 r-xp 00000000 fe:05 36176064 /lib/libaio.so.1.0.1 7f63660c8000-7f63662c7000 ---p 00001000 fe:05 36176064 /lib/libaio.so.1.0.1 7f63662c7000-7f63662c8000 rw-p 00000000 fe:05 36176064 /lib/libaio.so.1.0.1 7f63662c8000-7f63662cf000 r-xp 00000000 fe:05 36177639 /lib/librt-2.11.3.so 7f63662cf000-7f63664ce000 ---p 00007000 fe:05 36177639 /lib/librt-2.11.3.so 7f63664ce000-7f63664cf000 r--p 00006000 fe:05 36177639 /lib/librt-2.11.3.so 7f63664cf000-7f63664d0000 rw-p 00007000 fe:05 36177639 /lib/librt-2.11.3.so 7f63664d0000-7f6366550000 r-xp 00000000 fe:05 36177656 /lib/libm-2.11.3.so 7f6366550000-7f6366750000 ---p 00080000 fe:05 36177656 /lib/libm-2.11.3.so 7f6366750000-7f6366751000 r--p 00080000 fe:05 36177656 /lib/libm-2.11.3.so 7f6366751000-7f6366752000 rw-p 00081000 fe:05 36177656 /lib/libm-2.11.3.so 7f6366752000-7f6366754000 r-xp 00000000 fe:05 36177655 /lib/libdl-2.11.3.so 7f6366754000-7f6366954000 ---p 00002000 fe:05 36177655 /lib/libdl-2.11.3.so 7f6366954000-7f6366955000 r--p 00002000 fe:05 36177655 /lib/libdl-2.11.3.so 7f6366955000-7f6366956000 rw-p 00003000 fe:05 36177655 /lib/libdl-2.11.3.so 7f6366956000-7f636695e000 r-xp 00000000 fe:05 36177649 /lib/libcrypt-2.11.3.so 7f636695e000-7f6366b5d000 ---p 00008000 fe:05 36177649 /lib/libcrypt-2.11.3.so 7f6366b5d000-7f6366b5e000 r--p 00007000 fe:05 36177649 /lib/libcrypt-2.11.3.so 7f6366b5e000-7f6366b5f000 rw-p 00008000 fe:05 36177649 /lib/libcrypt-2.11.3.so 7f6366b5f000-7f6366b8d000 rw-p 00000000 00:00 0 7f6366b8d000-7f6366ba4000 r-xp 00000000 fe:05 36177637 /lib/libpthread-2.11.3.so 7f6366ba4000-7f6366da3000 ---p 00017000 fe:05 36177637 /lib/libpthread-2.11.3.so 7f6366da3000-7f6366da4000 r--p 00016000 fe:05 36177637 /lib/libpthread-2.11.3.so 7f6366da4000-7f6366da5000 rw-p 00017000 fe:05 36177637 /lib/libpthread-2.11.3.so 7f6366da5000-7f6366da9000 rw-p 00000000 00:00 0 7f6366da9000-7f6366dc7000 r-xp 00000000 fe:05 36177638 /lib/ld-2.11.3.so 7f6366e03000-7f6366e04000 ---p 00000000 00:00 0 7f6366e04000-7f6366e34000 rw-p 00000000 00:00 0 7f6366e34000-7f6366e35000 ---p 00000000 00:00 0 7f6366e35000-7f6366e65000 rw-p 00000000 00:00 0 7f6366e65000-7f6366e66000 ---p 00000000 00:00 0 7f6366e66000-7f6366e96000 rw-p 00000000 00:00 0 7f6366e96000-7f6366e97000 ---p 00000000 00:00 0 7f6366e97000-7f6366ec7000 rw-p 00000000 00:00 0 7f6366ec7000-7f6366ec8000 ---p 00000000 00:00 0 7f6366ec8000-7f6366ef8000 rw-p 00000000 00:00 0 7f6366ef8000-7f6366ef9000 ---p 00000000 00:00 0 7f6366ef9000-7f6366f29000 rw-p 00000000 00:00 0 7f6366f29000-7f6366f2a000 ---p 00000000 00:00 0 7f6366f2a000-7f6366f5a000 rw-p 00000000 00:00 0 7f6366f5a000-7f6366f5b000 ---p 00000000 00:00 0 7f6366f5b000-7f6366f8b000 rw-p 00000000 00:00 0 7f6366f8b000-7f6366f8c000 ---p 00000000 00:00 0 7f6366f8c000-7f6366fc1000 rw-p 00000000 00:00 0 7f6366fc3000-7f6366fc6000 rw-p 00000000 00:00 0 7f6366fc6000-7f6366fc7000 r--p 0001d000 fe:05 36177638 /lib/ld-2.11.3.so 7f6366fc7000-7f6366fc8000 rw-p 0001e000 fe:05 36177638 /lib/ld-2.11.3.so 7f6366fc8000-7f6366fc9000 rw-p 00000000 00:00 0 7fff8c72a000-7fff8c73f000 rw-p 00000000 00:00 0 [stack] 7fff8c78e000-7fff8c78f000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] 06:18:56 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=16777216 read_buffer_size=131072 max_used_connections=5 max_threads=800 thread_count=2 connection_count=1 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 6454484 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x609cc50 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f6366ec5ea8 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x35)[0x808a85] /usr/sbin/mysqld(handle_fatal_signal+0x403)[0x68fca3] /lib/libpthread.so.0(+0xeff0)[0x7f6366b9bff0] /lib/libc.so.6(gsignal+0x35)[0x7f6365d971b5] /lib/libc.so.6(abort+0x180)[0x7f6365d99fc0] /lib/libc.so.6(+0x6837b)[0x7f6365dcd37b] /lib/libc.so.6(+0x71bd6)[0x7f6365dd6bd6] /lib/libc.so.6(+0x74c6d)[0x7f6365dd9c6d] /lib/libc.so.6(__libc_malloc+0x70)[0x7f6365ddba30] /usr/sbin/mysqld(my_malloc+0x32)[0x8043f2] /usr/sbin/mysqld(_mi_write_blob_record+0x39)[0x7c05b9] /usr/sbin/mysqld(mi_write+0x828)[0x7da4a8] /usr/sbin/mysqld(_ZN9ha_myisam9write_rowEPh+0x43)[0x7a8183] /usr/sbin/mysqld(_ZN7handler12ha_write_rowEPh+0x5d)[0x694a0d] /usr/sbin/mysqld(_Z12write_recordP3THDP5TABLEP12st_copy_info+0x62)[0x57f512] /usr/sbin/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0xaff)[0x58640f] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x89f)[0x5951ef] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x33c)[0x59935c] /usr/sbin/mysqld(_ZN15Query_log_event14do_apply_eventEPK14Relay_log_infoPKcj+0xd5c)[0x73e4ac] /usr/sbin/mysqld(_Z26apply_event_and_update_posP9Log_eventP3THDP14Relay_log_info+0x125)[0x533555] /usr/sbin/mysqld[0x537ed1] /usr/sbin/mysqld(handle_slave_sql+0xa7b)[0x53a17b] /lib/libpthread.so.0(+0x68ca)[0x7f6366b938ca] /lib/libc.so.6(clone+0x6d)[0x7f6365e3492d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7f632849503b): is an invalid pointer Connection ID (thread ID): 530 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 120316 08:18:56 mysqld_safe Number of processes running now: 0 120316 08:18:56 mysqld_safe mysqld restarted 120316 8:18:57 [Note] Flashcache bypass: disabled 120316 8:18:57 [Note] Flashcache setup error is : open flash device failed 120316 8:18:57 [Warning] /usr/sbin/mysqld: ignoring option '--innodb-file-format-check' due to invalid value 'Barracuda' 120316 8:18:57 [Note] Plugin 'FEDERATED' is disabled. 120316 8:18:57 InnoDB: The InnoDB memory heap is disabled 120316 8:18:57 InnoDB: Mutexes and rw_locks use GCC atomic builtins 120316 8:18:57 InnoDB: Compressed tables use zlib 1.2.3 120316 8:18:57 InnoDB: Using Linux native AIO 120316 8:18:57 InnoDB: Initializing buffer pool, size = 500.0M 120316 8:18:57 InnoDB: Completed initialization of buffer pool 120316 8:18:57 InnoDB: highest supported file format is Barracuda. InnoDB: The log sequence number in ibdata files does not match InnoDB: the log sequence number in the ib_logfiles! 120316 8:18:57 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Error: the size of single-table tablespace file ./site@002dru@002dv4/analytics_categories.ibd InnoDB: is only 0 0, should be at least 65536!InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: In a MySQL replication slave the last master binlog file InnoDB: position 0 211179671, file name mysql-bin.000285 InnoDB: and relay log file InnoDB: position 0 142016370, file name /var/lib/mysql/relay-bin.000620 InnoDB: Last MySQL binlog file position 0 141599317, file name ./mysql-bin.000201 120316 8:18:59 InnoDB: Waiting for the background threads to start 120316 8:19:00 Percona XtraDB (http://www.percona.com) 1.1.8-rel24.1 started; log sequence number 1086208006520 120316 8:19:00 [Note] Recovering after a crash using mysql-bin 120316 8:19:00 [Note] Starting crash recovery... 120316 8:19:00 [Note] Crash recovery finished. 120316 8:19:01 [Warning] 'user' entry 'config_root@df30.dmz.alpari.dom' ignored in --skip-name-resolve mode. 120316 8:19:01 [Note] Event Scheduler: Loaded 0 events 120316 8:19:01 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.5.20-55-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Percona Server (GPL), Release 24.1 120316 8:19:01 [Note] Event Scheduler: scheduler thread started with id 1 120316 8:19:47 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000285' at position 211181822, relay log '/var/lib/mysql/relay-bin.000620' position: 142018521 120316 8:19:47 [Note] Slave I/O thread: connected to master 'replication@10.25.7.62:3306',replication started in log 'mysql-bin.000285' at position 211182594