MySQL do not stop because slave thread hunging

Reported by Marco Tusa on 2013-06-06
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Galera
Undecided
Alex Yurchenko

Bug Description

In a 5 nodes MySQL cluster using galera, with no activity MySQL remain hung and is not able to close clean a node when doing /etc/init.d/mysqld stop.

Checking the log it shows that is waiting to close and additional thread.

Below full information attached as requested by Alex in the email exchange.

130606 16:53:14 [Note] /usr/local/mysql/bin/mysqld: Normal shutdown

130606 16:53:14 [Note] WSREP: Stop replication
130606 16:53:14 [Note] WSREP: Provider disconnect
130606 16:53:14 [Note] WSREP: Closing send monitor...
130606 16:53:14 [Note] WSREP: Closed send monitor.
130606 16:53:14 [Note] WSREP: gcomm: terminating thread
130606 16:53:14 [Note] WSREP: gcomm: joining thread
130606 16:53:14 [Note] WSREP: Before Lock_thread_count
130606 16:53:14 [Note] WSREP: gcomm: closing backend
130606 16:53:15 [Note] WSREP: (5a4429b7-cee9-11e2-0800-722b31e72555, 'tcp://0.0.0.0:4567') turning message relay requesting off
130606 16:53:15 [Warning] WSREP: last inactive check more than PT1.5S ago, skipping check
130606 16:53:15 [Note] WSREP: view(view_id(NON_PRIM,5a4429b7-cee9-11e2-0800-722b31e72555,9) memb {
 5a4429b7-cee9-11e2-0800-722b31e72555,
} joined {
} left {
} partitioned {
 723e50fd-cec8-11e2-0800-672e9e87f901,
 8049192d-cee1-11e2-0800-8a871776ce2d,
 abd23113-cec8-11e2-0800-4b1406d6a7dc,
 ce08c7b4-cec7-11e2-0800-b441f087e5ff,
})
130606 16:53:15 [Note] WSREP: view((empty))
130606 16:53:15 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
130606 16:53:15 [Note] WSREP: gcomm: closed
130606 16:53:15 [Note] WSREP: Flow-control interval: [16, 16]
130606 16:53:15 [Note] WSREP: Received NON-PRIMARY.
130606 16:53:15 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 8)
130606 16:53:15 [Note] WSREP: Received self-leave message.
130606 16:53:15 [Note] WSREP: Flow-control interval: [0, 0]
130606 16:53:15 [Note] WSREP: Received SELF-LEAVE. Closing connection.
130606 16:53:15 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 8)
130606 16:53:15 [Note] WSREP: RECV thread exiting 0: Success
130606 16:53:15 [Note] WSREP: New cluster view: global state: 727b2c93-cec1-11e2-0800-4453b97f5bac:8, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 2
130606 16:53:15 [Note] WSREP: Setting wsrep_ready to 0
130606 16:53:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
130606 16:53:15 [Note] WSREP: New cluster view: global state: 727b2c93-cec1-11e2-0800-4453b97f5bac:8, view# -1: non-Primary, number of nodes: 0, my index: -1, protocol version 2
130606 16:53:15 [Note] WSREP: Setting wsrep_ready to 0
130606 16:53:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
130606 16:53:15 [Note] WSREP: recv_thread() joined.
130606 16:53:15 [Note] WSREP: Closing slave action queue.
130606 16:53:15 [Note] WSREP: applier thread exiting (code:0)
130606 16:53:15 [Note] WSREP: closing applier 1
130606 16:53:17 [Note] WSREP: waiting for client connections to close: 3

..... Hanging ....

Marco Tusa (marcotusa) wrote :
Marco Tusa (marcotusa) wrote :

Forgot to mention this happen with the Percona version AND using the code from codership directly.

the infor reported are from :
| version | 5.5.30-23.7.4-log |
| version_comment | Percona XtraDB Cluster (GPL) 5.5.30-23.7.4, Revision 405, wsrep_23.7.4.r3843

Alex Yurchenko (ayurchen) wrote :

Marco, what about SHOW FULL PROCESSLIST, SHOW GLOBAL STATUS and SHOW GLOBAL VARIABLES\G?

Specifically, how your configuration is different from defaults? And since it looks like there is some race, what hardware and distribution are you using?

Marco Tusa (marcotusa) wrote :

weird was not in the file loaded ...
here we go ... the other file

Marco Tusa (marcotusa) wrote :

ok is my fault ... :o( tooooo tired after a 18hrs spent in front of the console I mix the file ... this one should be the good one sorry about that ... I am just tired.

Marco Tusa (marcotusa) wrote :

Linux distribution is CentOS 6 64bit .

Running on VM

Alex Yurchenko (ayurchen) wrote :

Potential fix committed in r152

Changed in galera:
assignee: nobody → Alex Yurchenko (ayurchen)
status: New → Fix Committed
milestone: none → 23.2.6
Changed in galera:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers