Once again, the following day: 140902 7:16:30 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:32 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://10.0.1.11:4567 140902 7:16:34 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 34535308-319d-11e4-962e-367c0d11956c (tcp://10.0.1.11:4567), attempt 0 140902 7:16:34 [Warning] WSREP: last inactive check more than PT1.5S ago (PT4.20089S), skipping check 140902 7:16:34 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') cleaning up established 0x7fb69203d200 which is duplicate of 0x7fb652184400 140902 7:16:34 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:34 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:34 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:34 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:34 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') turning message relay requesting off 140902 7:16:41 [Warning] WSREP: last inactive check more than PT1.5S ago (PT6.86932S), skipping check 140902 7:16:41 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:42 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://10.0.1.11:4567 tcp://10.0.1.5:4567 140902 7:16:42 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 34535308-319d-11e4-962e-367c0d11956c (tcp://10.0.1.11:4567), attempt 0 140902 7:16:42 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 71da7145-3082-11e4-a9cb-d346363304ed (tcp://10.0.1.5:4567), attempt 0 140902 7:16:44 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 34535308-319d-11e4-962e-367c0d11956c (tcp://10.0.1.11:4567), attempt 0 140902 7:16:44 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 71da7145-3082-11e4-a9cb-d346363304ed (tcp://10.0.1.5:4567), attempt 0 140902 7:16:45 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 34535308-319d-11e4-962e-367c0d11956c (tcp://10.0.1.11:4567), attempt 0 140902 7:16:45 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 71da7145-3082-11e4-a9cb-d346363304ed (tcp://10.0.1.5:4567), attempt 0 140902 7:16:46 [Note] WSREP: evs::proto(451a0e59-3188-11e4-8945-026d25e51b59, OPERATIONAL, view_id(REG,34535308-319d-11e4-962e-367c0d11956c,30)) suspecting node: 34535308-319d-11e4-962e-367c0d11956c 140902 7:16:46 [Note] WSREP: evs::proto(451a0e59-3188-11e4-8945-026d25e51b59, OPERATIONAL, view_id(REG,34535308-319d-11e4-962e-367c0d11956c,30)) suspecting node: 71da7145-3082-11e4-a9cb-d346363304ed 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 34535308-319d-11e4-962e-367c0d11956c (tcp://10.0.1.11:4567), attempt 0 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') reconnecting to 71da7145-3082-11e4-a9cb-d346363304ed (tcp://10.0.1.5:4567), attempt 0 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') turning message relay requesting off 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') cleaning up duplicate 0x7fb65207b680 after established 0x7fb69203d200 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:47 [Note] WSREP: (451a0e59-3188-11e4-8945-026d25e51b59, 'tcp://0.0.0.0:4567') address 'tcp://10.0.1.10:4567' pointing to uuid 451a0e59-3188-11e4-8945-026d25e51b59 is blacklisted, skipping 140902 7:16:47 [Note] WSREP: view(view_id(NON_PRIM,34535308-319d-11e4-962e-367c0d11956c,30) memb { 451a0e59-3188-11e4-8945-026d25e51b59,0 } joined { } left { } partitioned { 34535308-319d-11e4-962e-367c0d11956c,0 71da7145-3082-11e4-a9cb-d346363304ed,0 }) 140902 7:16:47 [Note] WSREP: view(view_id(NON_PRIM,451a0e59-3188-11e4-8945-026d25e51b59,31) memb { 451a0e59-3188-11e4-8945-026d25e51b59,0 } joined { } left { } partitioned { 34535308-319d-11e4-962e-367c0d11956c,0 71da7145-3082-11e4-a9cb-d346363304ed,0 }) 140902 7:16:47 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 140902 7:16:48 [Note] WSREP: Flow-control interval: [16, 16] 140902 7:16:48 [Note] WSREP: Received NON-PRIMARY. 140902 7:16:48 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 2140112) 140902 7:16:48 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 140902 7:16:48 [Note] WSREP: Flow-control interval: [16, 16] 140902 7:16:48 [Note] WSREP: Received NON-PRIMARY. 140902 7:16:48 [Note] WSREP: declaring 34535308-319d-11e4-962e-367c0d11956c stable 140902 7:16:48 [Note] WSREP: declaring 71da7145-3082-11e4-a9cb-d346363304ed stable 140902 7:16:48 [Warning] WSREP: Send action {(nil), 1140, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Note] WSREP: Node 34535308-319d-11e4-962e-367c0d11956c state prim 140902 7:16:48 [Warning] WSREP: Send action {(nil), 360, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Warning] WSREP: Send action {(nil), 360, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Warning] WSREP: Send action {(nil), 360, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Warning] WSREP: Send action {(nil), 360, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Warning] WSREP: Send action {(nil), 829, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Warning] WSREP: Send action {(nil), 360, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Warning] WSREP: Send action {(nil), 24870, TORDERED} returned -107 (Transport endpoint is not connected) 140902 7:16:48 [Note] WSREP: view(view_id(PRIM,34535308-319d-11e4-962e-367c0d11956c,32) memb { 34535308-319d-11e4-962e-367c0d11956c,0 451a0e59-3188-11e4-8945-026d25e51b59,0 71da7145-3082-11e4-a9cb-d346363304ed,0 } joined { } left { } partitioned { }) 140902 7:16:48 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3 140902 7:16:48 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 140902 7:16:48 [Note] WSREP: STATE EXCHANGE: sent state msg: 1be1f9ff-3271-11e4-a10b-eb332eb0afab 140902 7:16:48 [Note] WSREP: STATE EXCHANGE: got state msg: 1be1f9ff-3271-11e4-a10b-eb332eb0afab from 0 () 140902 7:16:48 [Note] WSREP: STATE EXCHANGE: got state msg: 1be1f9ff-3271-11e4-a10b-eb332eb0afab from 2 () 140902 7:16:48 [Note] WSREP: STATE EXCHANGE: got state msg: 1be1f9ff-3271-11e4-a10b-eb332eb0afab from 1 () 140902 7:16:48 [Note] WSREP: Quorum results: version = 3, component = PRIMARY, conf_id = 30, members = 2/3 (joined/total), act_id = 2140138, last_appl. = 2140029, protocols = 0/5/3 (gcs/repl/appl), group UUID = 40179a77-2bf4-11e4-b46a-a30695b570ce 140902 7:16:48 [Note] WSREP: Flow-control interval: [28, 28] 140902 7:16:48 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 2140138) 140902 7:16:48 [Note] WSREP: New cluster view: global state: 40179a77-2bf4-11e4-b46a-a30695b570ce:2140112, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3 140902 7:16:48 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 140902 7:16:48 [Note] WSREP: New cluster view: global state: 40179a77-2bf4-11e4-b46a-a30695b570ce:2140112, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3 140902 7:16:48 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 140902 7:16:48 [Note] WSREP: State transfer required: Group state: 40179a77-2bf4-11e4-b46a-a30695b570ce:2140138 Local state: 40179a77-2bf4-11e4-b46a-a30695b570ce:2140112 140902 7:16:48 [Note] WSREP: New cluster view: global state: 40179a77-2bf4-11e4-b46a-a30695b570ce:2140138, view# 31: Primary, number of nodes: 3, my index: 1, protocol version 3 140902 7:16:48 [Warning] WSREP: Gap in state sequence. Need state transfer. 140902 7:16:50 [Note] WSREP: You have configured 'xtrabackup-v2' state snapshot transfer method which cannot be performed on a running server. Wsrep provider won't be able to fall back to it if other means of state transfer are unavailable. In that case you will need to restart the server. 140902 7:16:50 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 140902 7:16:50 [Note] WSREP: REPL Protocols: 5 (3, 1) 140902 7:16:50 [Note] WSREP: Service thread queue flushed. 140902 7:16:50 [Note] WSREP: Assign initial position for certification: 2140138, protocol version: 3 140902 7:16:50 [Note] WSREP: Service thread queue flushed. 140902 7:16:50 [Warning] WSREP: Releasing seqno 2140112 before 2140113 was assigned. 140902 7:16:50 [Warning] WSREP: Failed to prepare for incremental state transfer: Failed to open IST listener at tcp://10.0.1.10:4568', asio error 'Already open.': 1 (Operation not permitted) at galera/src/ist.cpp:prepare():334. IST will be unavailable. 140902 7:16:50 [Note] WSREP: Member 1.0 () requested state transfer from '*any*'. Selected 0.0 ()(SYNCED) as donor. 140902 7:16:50 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 2140141) 140902 7:16:50 [Note] WSREP: Requesting state transfer: success, donor: 0 140902 7:16:50 [Note] WSREP: GCache DEBUG: RingBuffer::seqno_reset(): discarded 0 bytes 140902 7:16:50 [Warning] WSREP: 0.0 (): State transfer to 1.0 () failed: -125 (Operation canceled) 140902 7:16:50 [ERROR] WSREP: gcs/src/gcs_group.c:gcs_group_handle_join_msg():723: Will never receive state. Need to abort. 140902 7:16:50 [Note] WSREP: gcomm: terminating thread 140902 7:16:50 [Note] WSREP: gcomm: joining thread 140902 7:16:50 [Note] WSREP: gcomm: closing backend 140902 7:16:50 [Note] WSREP: view(view_id(NON_PRIM,34535308-319d-11e4-962e-367c0d11956c,32) memb { 451a0e59-3188-11e4-8945-026d25e51b59,0 } joined { } left { } partitioned { 34535308-319d-11e4-962e-367c0d11956c,0 71da7145-3082-11e4-a9cb-d346363304ed,0 }) 140902 7:16:50 [Note] WSREP: GCache DEBUG: RingBuffer::seqno_reset(): found 1/3 locked buffers 140902 7:16:50 [Note] WSREP: view((empty)) 140902 7:16:50 [Note] WSREP: gcomm: closed 140902 7:16:50 [Note] WSREP: /usr/sbin/mysqld: Terminated.