WSREP: FSM: no such a transition COMMITTING -> REPLICATING
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
MySQL patches by Codership |
Fix Released
|
Low
|
Seppo Jaakola | ||
Percona XtraDB Cluster moved to https://jira.percona.com/projects/PXC |
Fix Released
|
Undecided
|
Unassigned | ||
5.5 |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
The mysqld process crashes after only a few queries. From this error, it's impossible to see what went wrong. Could be a bug, could be an error on our end, but there's no way to tell.
Setup:
3 - node cluster (test setup)
All on virtual machines 2 CPU / 2 GB (node3, 4 & 5)
All InnoDB
Relevant my.cnf parts: (taken from node3)
-------
# CACHES AND LIMITS #
tmp_table_size = 32M
max_heap_table_size = 32M
query_cache_type = 0
query_cache_size = 0
max_connections = 200
thread_cache_size = 20
open_files_limit = 65535
table_definitio
table_open_cache = 4096
# INNODB #
innodb_flush_method = O_DIRECT
innodb_
innodb_
innodb_
innodb_
innodb_
server_id=1
wsrep_cluster_
wsrep_provider=
wsrep_provider_
wsrep_slave_
wsrep_cluster_
wsrep_sst_
wsrep_sst_
wsrep_node_
wsrep_node_
wsrep_node_
innodb_
innodb_
-------
Crash happens also when running only 1 node
Percona Xtradb cluster Version 5.5.28
*no* messages of any importance before first message in trace below
Information about "no such a transition COMMITTING -> REPLICATING" nowhere to be found (manual, forum, documentation, google) but that could also be a lack of searching skills on my end.
=======
121210 9:52:52 [ERROR] WSREP: FSM: no such a transition COMMITTING -> REPLICATING
08:52:52 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://
key_buffer_
read_buffer_
max_used_
max_threads=200
thread_count=6
connection_count=6
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x520a080
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f802ede8e20 thread_stack 0x40000
mysqld(
mysqld(
/lib/x86_
/lib/x86_
/lib/x86_
/usr/lib/
/usr/lib/
/usr/lib/
mysqld(
mysqld[0x6689fb]
mysqld(
mysqld(
mysqld(
mysqld(
mysqld[0x5b2b62]
mysqld(
mysqld(
mysqld(
mysqld(
/lib/x86_
/lib/x86_
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f7f90004c50): is an invalid pointer
Connection ID (thread ID): 44
Status: NOT_KILLED
=======
affects: | percona-server → percona-xtradb-cluster |
Changed in percona-xtradb-cluster: | |
status: | New → Confirmed |
Changed in codership-mysql: | |
status: | New → Confirmed |
Changed in codership-mysql: | |
assignee: | nobody → Seppo Jaakola (seppo-jaakola) |
milestone: | none → 5.5.28-23.8 |
Changed in codership-mysql: | |
status: | Confirmed → In Progress |
Changed in percona-xtradb-cluster: | |
milestone: | none → 5.5.29-23.8 |
Changed in percona-xtradb-cluster: | |
milestone: | 5.5.29-23.8 → 5.5.29-27.3.1 |
status: | Confirmed → Fix Committed |
milestone: | 5.5.29-27.3.1 → 5.5.29-23.7.1 |
Changed in codership-mysql: | |
milestone: | 5.5.29-24.8 → 5.5.29-23.7.1 |
Changed in percona-xtradb-cluster: | |
status: | Fix Committed → Fix Released |
no longer affects: | percona-xtradb-cluster/5.6 |
Extra information: it seems like this is only happening when running queries via PHP. When executing exactly the same set of queries using cat queries.sql | mysql -u root -p -D etc etc nothing goes wrong.
Following packages installed on Ubuntu 12.04 LTS:
percona-xtrabackup xtradb- cluster- client- 5.5 xtradb- cluster- common- 5.5 xtradb- cluster- galera- 2.x xtradb- cluster- server- 5.5
percona-
percona-
percona-
percona-
<guessing>
It looks like this is one of the tables which, when a record is inserted via PHP, causes the crash. We've seen several different queries on which the server crashes, but it looks like it's limited to 3 tables. This is one of them:
CREATE TABLE `CRMentityformc ache` ( last_change` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`formcacheid` mediumint(9) NOT NULL AUTO_INCREMENT,
`eid` mediumint(9) NOT NULL,
`formid` mediumint(9) NOT NULL,
`tabletype` varchar(15) NOT NULL DEFAULT 'entity',
`user` mediumint(9) NOT NULL,
`content` varchar(255) DEFAULT NULL,
`timestamp_
PRIMARY KEY (`formcacheid`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8 COMMENT='Interleave Cached parsed forms'
Application log show the last query executed which (probably) was directly causing the crash. Again, when executing this on the mysql prompt, all is ok and via PHP this query doesn't ALWAYS crash the server - only about 40-60% of the time (2 to 3 pageloads).
SQL: ERROR_LAST_ QUERY_WENT_ WRONG Lost connection to MySQL server during query query : DELETE FROM CRMentityformcache WHERE user='1'
Of course the tables are checked and ok and the software / queries / etc run fine on a stock MySQL server.
</guessing>