I can quite reliably crash MySQL by running 'nmap' against the server running it, but only as a non-root user. If I run nmap as root, all's well. I think the difference lies in how nmap scans ports when run as root vs. non-root. Some strace output:
as root, it uses "sendto()":
sendto(3, "E\0\0,\260x\0\0(\6\211#\212H\204\320\212H\277\317\364v\21\327\261\"?\313\0\0\0\0"..., 44, 0, {sa_family=AF_INET, sin_port=htons(4567), sin_addr=inet_addr("xxx.xxx.xxx.xxx")}, 16) = 44
as non-root, it uses "connect()":
connect(3, {sa_family=AF_INET, sin_port=htons(4567), sin_addr=inet_addr("xxx.xxx.xxx.xxx")}, 16) = -1 EINPROGRESS (Operation now in progress)
(I don't know that it's the connection to port 4567 that is the culprit; just used those lines from strace output as examples).
MySQL dies like so:
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<asio::system_error> >'
what(): Transport endpoint is not connected
22:47:10 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=11
max_threads=2002
thread_count=9
connection_count=9
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4389869 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x35)[0x7d0765]
/usr/sbin/mysqld(handle_fatal_signal+0x4b4)[0x6a9c64]
/lib64/libpthread.so.0(+0xf520)[0x7f0ddfe05520]
/lib64/libc.so.6(gsignal+0x35)[0x7f0dde9dba45]
/lib64/libc.so.6(abort+0x175)[0x7f0dde9dd225]
/usr/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d)[0x7f0ddc13ca7d]
/usr/lib64/libstdc++.so.6(+0xbcc06)[0x7f0ddc13ac06]
/usr/lib64/libstdc++.so.6(+0xbcc33)[0x7f0ddc13ac33]
/usr/lib64/libstdc++.so.6(+0xbcd2e)[0x7f0ddc13ad2e]
/usr/lib64/libgalera_smm.so(_ZN5boost15throw_exceptionIN4asio12system_errorEEEvRKT_+0x18a)[0x7f0ddc4a93ca]
/usr/lib64/libgalera_smm.so(_ZN4asio6detail11throw_errorERKNS_10error_codeE+0x5b)[0x7f0ddc4a94eb]
/usr/lib64/libgalera_smm.so(_ZNK4asio12basic_socketINS_2ip3tcpENS_21stream_socket_serviceIS2_EEE15remote_endpointEv+0xc1)[0x7f0ddc4a9671]
/usr/lib64/libgalera_smm.so(_ZN5gcomm13AsioTcpSocket18assign_remote_addrEv+0x5e7)[0x7f0ddc49a657]
/usr/lib64/libgalera_smm.so(_ZN5gcomm15AsioTcpAcceptor14accept_handlerEN5boost10shared_ptrINS_6SocketEEERKN4asio10error_codeE+0xc4)[0x7f0ddc49b0e4]
/usr/lib64/libgalera_smm.so(_ZN4asio6detail25reactive_socket_accept_opINS_12basic_socketINS_2ip3tcpENS_21stream_socket_serviceIS4_EEEES4_N5boost3_bi6bind_tIvNS8_4_mfi3mf2IvN5gcomm15AsioTcpAcceptorENS8_10shared_ptrINSD_6SocketEEERKNS_10error_codeEEENS9_5list3INS9_5valueIPSE_EENSN_ISH_EEPFNS8_3argILi1EEEvEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESI_m+0x180)[0x7f0ddc4a4eb0]
/usr/lib64/libgalera_smm.so(_ZN4asio6detail15task_io_service3runERNS_10error_codeE+0x459)[0x7f0ddc4c68c9]
/usr/lib64/libgalera_smm.so(_ZN5gcomm12AsioProtonet10event_loopERKN2gu8datetime6PeriodE+0x1d6)[0x7f0ddc4be816]
/usr/lib64/libgalera_smm.so(_ZN9GCommConn3runEv+0x57)[0x7f0ddc4d8777]
/usr/lib64/libgalera_smm.so(_ZN9GCommConn6run_fnEPv+0x9)[0x7f0ddc4dd469]
/lib64/libpthread.so.0(+0x77e1)[0x7f0ddfdfd7e1]
/lib64/libc.so.6(clone+0x6d)[0x7f0ddea8f8ed]
You may download the Percona Server operations manual by visiting http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
130524 15:47:10 mysqld_safe Number of processes running now: 0
130524 15:47:10 mysqld_safe WSREP: not restarting wsrep node automatically
130524 15:47:10 mysqld_safe mysqld from pid file /mnt/ssd/mysql/xxx.pid ended
I saw this discussion of a very similar problem on the "Percona discussions" google group, but I'm sure an actual bug was ever submitted - a search on this site didn't find anything, but I apologize if this ends up being a duplicate.
Basic info:
# uname -a 235.el6. x86_64 #1 SMP Fri Feb 17 11:48:53 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
Linux 2.6.32-
# rpm -qa | grep -i percona xtrabackup- 2.0.7-552. rhel6.x86_ 64 XtraDB- Cluster- shared- 5.5.30- 23.7.4. 406.rhel6. x86_64 toolkit- 2.1.4-1. noarch XtraDB- Cluster- client- 5.5.30- 23.7.4. 406.rhel6. x86_64 Server- shared- compat- 5.5.30- rel30.2. 508.rhel6. x86_64 XtraDB- Cluster- server- 5.5.30- 23.7.4. 406.rhel6. x86_64 XtraDB- Cluster- galera- 2.5-1.150. rhel6.x86_ 64
percona-
Percona-
percona-
Percona-
Percona-
Percona-
Percona-
I can quite reliably crash MySQL by running 'nmap' against the server running it, but only as a non-root user. If I run nmap as root, all's well. I think the difference lies in how nmap scans ports when run as root vs. non-root. Some strace output:
as root, it uses "sendto()": \260x\0\ 0(\6\211# \212H\204\ 320\212H\ 277\317\ 364v\21\ 327\261\ "?\313\ 0\0\0\0" ..., 44, 0, {sa_family=AF_INET, sin_port= htons(4567) , sin_addr= inet_addr( "xxx.xxx. xxx.xxx" )}, 16) = 44
sendto(3, "E\0\0,
as non-root, it uses "connect()": htons(4567) , sin_addr= inet_addr( "xxx.xxx. xxx.xxx" )}, 16) = -1 EINPROGRESS (Operation now in progress)
connect(3, {sa_family=AF_INET, sin_port=
(I don't know that it's the connection to port 4567 that is the culprit; just used those lines from strace output as examples).
MySQL dies like so:
terminate called after throwing an instance of 'boost: :exception_ detail: :clone_ impl<boost: :exception_ detail: :error_ info_injector< asio::system_ error> >' bugs.percona. com/
what(): Transport endpoint is not connected
22:47:10 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://
key_buffer_ size=8388608 size=131072 connections= 11 size)*max_ threads = 4389869 K bytes of memory
read_buffer_
max_used_
max_threads=2002
thread_count=9
connection_count=9
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0 mysqld( my_print_ stacktrace+ 0x35)[0x7d0765] mysqld( handle_ fatal_signal+ 0x4b4)[ 0x6a9c64] libpthread. so.0(+0xf520) [0x7f0ddfe05520 ] libc.so. 6(gsignal+ 0x35)[0x7f0dde9 dba45] libc.so. 6(abort+ 0x175)[ 0x7f0dde9dd225] libstdc+ +.so.6( _ZN9__gnu_ cxx27__ verbose_ terminate_ handlerEv+ 0x12d)[ 0x7f0ddc13ca7d] libstdc+ +.so.6( +0xbcc06) [0x7f0ddc13ac06 ] libstdc+ +.so.6( +0xbcc33) [0x7f0ddc13ac33 ] libstdc+ +.so.6( +0xbcd2e) [0x7f0ddc13ad2e ] libgalera_ smm.so( _ZN5boost15thro w_exceptionIN4a sio12system_ errorEEEvRKT_ +0x18a) [0x7f0ddc4a93ca ] libgalera_ smm.so( _ZN4asio6detail 11throw_ errorERKNS_ 10error_ codeE+0x5b) [0x7f0ddc4a94eb ] libgalera_ smm.so( _ZNK4asio12basi c_socketINS_ 2ip3tcpENS_ 21stream_ socket_ serviceIS2_ EEE15remote_ endpointEv+ 0xc1)[0x7f0ddc4 a9671] libgalera_ smm.so( _ZN5gcomm13Asio TcpSocket18assi gn_remote_ addrEv+ 0x5e7)[ 0x7f0ddc49a657] libgalera_ smm.so( _ZN5gcomm15Asio TcpAcceptor14ac cept_handlerEN5 boost10shared_ ptrINS_ 6SocketEEERKN4a sio10error_ codeE+0xc4) [0x7f0ddc49b0e4 ] libgalera_ smm.so( _ZN4asio6detail 25reactive_ socket_ accept_ opINS_12basic_ socketINS_ 2ip3tcpENS_ 21stream_ socket_ serviceIS4_ EEEES4_ N5boost3_ bi6bind_ tIvNS8_ 4_mfi3mf2IvN5gc omm15AsioTcpAcc eptorENS8_ 10shared_ ptrINSD_ 6SocketEEERKNS_ 10error_ codeEEENS9_ 5list3INS9_ 5valueIPSE_ EENSN_ISH_ EEPFNS8_ 3argILi1EEEvEEE EEE11do_ completeEPNS0_ 15task_ io_serviceEPNS0 _25task_ io_service_ operationESI_ m+0x180) [0x7f0ddc4a4eb0 ] libgalera_ smm.so( _ZN4asio6detail 15task_ io_service3runE RNS_10error_ codeE+0x459) [0x7f0ddc4c68c9 ] libgalera_ smm.so( _ZN5gcomm12Asio Protonet10event _loopERKN2gu8da tetime6PeriodE+ 0x1d6)[ 0x7f0ddc4be816] libgalera_ smm.so( _ZN9GCommConn3r unEv+0x57) [0x7f0ddc4d8777 ] libgalera_ smm.so( _ZN9GCommConn6r un_fnEPv+ 0x9)[0x7f0ddc4d d469] libpthread. so.0(+0x77e1) [0x7f0ddfdfd7e1 ] libc.so. 6(clone+ 0x6d)[0x7f0ddea 8f8ed] www.percona. com/software/ percona- server/. You may find information mysql/xxx. pid ended
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
/usr/sbin/
/usr/sbin/
/lib64/
/lib64/
/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/usr/lib64/
/lib64/
/lib64/
You may download the Percona Server operations manual by visiting
http://
in the manual which will help you identify the cause of the crash.
130524 15:47:10 mysqld_safe Number of processes running now: 0
130524 15:47:10 mysqld_safe WSREP: not restarting wsrep node automatically
130524 15:47:10 mysqld_safe mysqld from pid file /mnt/ssd/
I saw this discussion of a very similar problem on the "Percona discussions" google group, but I'm sure an actual bug was ever submitted - a search on this site didn't find anything, but I apologize if this ends up being a duplicate.
https:/ /groups. google. com/forum/ ?fromgroups# !topic/ percona- discussion/ gUWvXOKR_ 88
Thanks!