Comment 5 for bug 1232747

I attached the logs. There definitely was some sort of network event that preceded the crash and exceptions.

Also, all 3 nodes quit at the same time. dbe0's timezone was set to US Central instead of UTC so the hours don't match up but the minutes/seconds do.

Our cluster is comprised of 3 WAN nodes which are hosted in separate data centers.

dbw0 - (Dallas, TX Linode) - does reads/writes, 20ms latency to dbc0, 40ms latency to dbe0
dbc0 - (Atlanta, GA Linode) - only does mysqldump backups, 20ms latency to both dbe0/dbw0
dbe0 - (Newark, NJ Linode) - does reads/writes, 20ms latency to dbc0, 40ms latency to dbw0

Cluster was created 5 weeks ago.

dbe0 had been running percona-xtradb-cluster-server-5.5 5.5.31-23.7.5-438.raring for the whole 5 weeks.
dbc0 had been running for 4.5 days (was upgraded and restarted)
dbw0 had been running for 5.5 days (was upgraded and restarted)

We have had network hiccups before but they usually result in one node disappearing for a short time and then re-joining without incident.

There are a couple of "[Warning] WSREP: Quorum: No node with complete state:" warnings in the logs I attached which we haven't seen before.