System lock without explanation

[Daniel Ylitalo]

We are running a 4 node cluster and it just locked up for about 20 minutes.

Three nodes had a system thread that had status "System lock" and one did not, like this:
338 system user NULL Sleep 1402 System lock NULL 0 0

How do i find out what made the "System lock"?

I have the full proclist of the node that did not have the "System lock" thread, but there is no update/delete/alter queries that might explain why the three other nodes locked up.

Running:
5.6.22-72.0-56-log - Percona XtraDB Cluster (GPL), Release rel72.0, Revision 978, WSREP version 25.8, wsrep_25.8.r4150
Ubuntu Server 14.04.2

[Nilnandan Joshi (nilnandan-joshi)]

Can you provide the show engine innodb status\G, show status like 'ws%' and error logs from all nodes?
Btw: are you using LOAD DATA to insert records?

[Daniel Ylitalo (danigl)]

Error logs coming in a moment.

We are not running any LOAD DATA calls

[Daniel Ylitalo (danigl)]

Unfortunately we dont have the error log for the 3rd node, seems some has removed it and forgot to signal mysql to reload the logs

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-1824