Crash in EC2 node

Bug #914976 reported by Percona
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Galera
Fix Released
High
Alex Yurchenko
Percona XtraDB Cluster moved to https://jira.percona.com/projects/PXC
Fix Released
Undecided
Unassigned

Bug Description

Starting Percona XtraDB Cluster with following option cause crash

cat /etc/my.cnf
[mysqld]
datadir=/mnt/data/mysql
user=mysql

log_error=error.log

binlog_format=ROW

wsrep_provider=/usr/lib64/libgalera_smm.so

wsrep_cluster_address=gcomm://ec2-79-125-75-115.eu-west-1.compute.amazonaws.com

wsrep_sst_receive_address=ec2-175-41-152-75.ap-southeast-1.compute.amazonaws.com
wsrep_provider_options = "ec2-175-41-152-75.ap-southeast-1.compute.amazonaws.com; ist.recv_addr=ec2-175-41-152-75.ap-southeast-1.compute.amazonaws.com"

wsrep_slave_threads=2
wsrep_cluster_name=trimethylxanthine
wsrep_sst_method=rsync
wsrep_node_name=node2

innodb_locks_unsafe_for_binlog=1
innodb_autoinc_lock_mode=2

innodb_log_file_size=64M

log file:
120111 14:31:20 mysqld_safe Starting mysqld daemon with databases from /mnt/data/mysql
120111 14:31:20 [Note] Flashcache bypass: disabled
120111 14:31:20 [Note] Flashcache setup error is : ioctl failed

120111 14:31:20 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/libgalera_smm.so'
120111 14:31:20 [Note] WSREP: wsrep_load(): Galera 2.0beta(r103) by Codership Oy <email address hidden> loaded succesfully.
terminate called after throwing an instance of 'std::length_error'
  what(): basic_string::_S_create
120111 14:31:21 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=0
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 330456 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = (nil) thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7b7c69]
/usr/sbin/mysqld(handle_segfault+0x464)[0x514094]
/lib64/libpthread.so.0(+0xf4a0)[0x7f9d97fa34a0]
/lib64/libc.so.6(gsignal+0x35)[0x7f9d9715d885]
/lib64/libc.so.6(abort+0x175)[0x7f9d9715f065]
/usr/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d)[0x7f9d957f2a7d]
/usr/lib64/libstdc++.so.6(+0xbcc06)[0x7f9d957f0c06]
/usr/lib64/libstdc++.so.6(+0xbcc33)[0x7f9d957f0c33]
/usr/lib64/libstdc++.so.6(+0xbcc46)[0x7f9d957f0c46]
/usr/lib64/libstdc++.so.6(__cxa_call_unexpected+0x43)[0x7f9d957f02d3]
/usr/lib64/libgalera_smm.so(_ZN2gu6Config5parseERSt3mapISsSsSt4lessISsESaISt4pairIKSsSsEEERS5_+0x923)[0x7f9d95d18983]
/usr/lib64/libgalera_smm.so(_ZN2gu6ConfigC2ERKSs+0x3e)[0x7f9d95d18b5e]
/usr/lib64/libgalera_smm.so(_ZN6galera13ReplicatorSMMC1EPK15wsrep_init_args+0x5f)[0x7f9d95e31d4f]
/usr/lib64/libgalera_smm.so(galera_init+0x34)[0x7f9d95e46594]
/usr/sbin/mysqld(_Z10wsrep_initv+0x22e)[0x64ed8e]
/usr/sbin/mysqld(_Z18wsrep_init_startupb+0x10)[0x64ef90]
/usr/sbin/mysqld[0x51c517]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0xac1)[0x51fb51]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f9d97149cdd]
/usr/sbin/mysqld[0x512929]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

Related branches

Revision history for this message
Percona (percona-team) wrote :

it crashed because option
wsrep_provider_options
has wrong format there.
it should be

wsrep_provider_options = "gmcast.listen_addr=tcp://ec2-175-41-152-75.ap-southeast-1.compute.amazonaws.com; ist.recv_addr=ec2-175-41-152-75.ap-southeast-1.compute.amazonaws.com"

But it is not a valid reason to crash :)

Changed in codership-mysql:
status: New → In Progress
assignee: nobody → Seppo Jaakola (seppo-jaakola)
Changed in galera:
assignee: nobody → Alex Yurchenko (ayurchen)
importance: Undecided → High
milestone: none → 22.1.2
status: New → Confirmed
no longer affects: codership-mysql
Changed in galera:
status: Confirmed → Fix Committed
Changed in galera:
status: Fix Committed → Fix Released
Changed in percona-xtradb-cluster:
status: New → Fix Released
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-1199

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.