_mysql_exceptions.OperationalError: (2013, 'Lost connection to MySQL server during query')

[Jason Hobbs]

Our HA deployment of percona cluster failed with this error:

http://paste.ubuntu.com/26526195/

Crashdump is attached, bundle: http://paste.ubuntu.com/26526202/

[Liam Young (gnuoy)]

This looks to me like an underlying network issue. The error in the charm occurred at 08:16:06. The mysql/0:/var/log/mysql/error.log shows mysql terminating at this time with an error that appears to be synonymous with a split brain ( https://severalnines.com/blog/become-mysql-dba-blog-series-troubleshooting-galera-cluster-issues-part-1 ).

error.log message:

2018-02-05 08:16:06 106885 [Note] WSREP: evs::proto(5d3d1403, LEAVING, view_id(REG,5d3d1403,6)) suspecting node: c06cc037
2018-02-05 08:16:06 106885 [Note] WSREP: evs::proto(5d3d1403, LEAVING, view_id(REG,5d3d1403,6)) suspected node without join message, declaring inactive
2018-02-05 08:16:06 106885 [Note] WSREP: view(view_id(NON_PRIM,5d3d1403,6) memb {
 5d3d1403,0
} joined {
} left {
} partitioned {
 c06cc037,0
})
2018-02-05 08:16:06 106885 [Note] WSREP: view((empty))
2018-02-05 08:16:06 106885 [Note] WSREP: gcomm: closed
2018-02-05 08:16:06 106885 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2018-02-05 08:16:06 106885 [Note] WSREP: Flow-control interval: [16, 16]
2018-02-05 08:16:06 106885 [Note] WSREP: Trying to continue unpaused monitor
2018-02-05 08:16:06 106885 [Note] WSREP: Received NON-PRIMARY.
2018-02-05 08:16:06 106885 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 909)
2018-02-05 08:16:06 106885 [Note] WSREP: Received self-leave message.
2018-02-05 08:16:06 106885 [Note] WSREP: Flow-control interval: [0, 0]
2018-02-05 08:16:06 106885 [Note] WSREP: Trying to continue unpaused monitor
2018-02-05 08:16:06 106885 [Note] WSREP: Received SELF-LEAVE. Closing connection.
2018-02-05 08:16:06 106885 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 909)
2018-02-05 08:16:06 106885 [Note] WSREP: RECV thread exiting 0: Success
2018-02-05 08:16:06 106885 [Note] WSREP: recv_thread() joined.
2018-02-05 08:16:06 106885 [Note] WSREP: Closing replication queue.
2018-02-05 08:16:06 106885 [Note] WSREP: Closing slave action queue.
2018-02-05 08:16:06 106885 [Note] WSREP: /usr/sbin/mysqld: Terminated.

[Liam Young (gnuoy)]

I think https://bugs.launchpad.net/charm-percona-cluster/+bug/1746524/comments/6 applies to this bug too

[Chris MacNaughton (chris.macnaughton)]

Given the lack of updates or response to @gnuoy's observation about networking / percona network partitioning, I'm marking this as incomplete unless we have a new reproducer where we don't see these issues.

[Launchpad Janitor (janitor)]

[Expired for OpenStack percona-cluster charm because there has been no activity for 60 days.]